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A Message From 
the Auditor General of Canada 


Accountability for security and intelligence activities: 
A new challenge for Parliament 


Security and intelligence activity in the Government of Canada has 
expanded considerably in recent years, particularly since September 11, 2001. 
The federal government now spends more than $1 billion a year on 

these activities. 


Parliament needs objective information to determine whether public funds 
have been spent properly and managed well. For most activities of 
government, it is my job as Auditor General to provide that information in 
our performance audit reports. 


Security and intelligence activities, however, create special problems for our 
normal process of providing information to Parliament and, once tabled, to 
the wider public. By their very nature, many aspects of security and 
intelligence activities must remain secret. 


This dilemma became very real to us when we were preparing one of the 
chapters in this report. Chapter 2 on the 2001 anti-terrorism initiative deals 
with certain aspects of the air passenger security system managed by 
Transport Canada and the Canadian Air Transport Security Authority. 


To determine the effectiveness of the passenger security system, Transport 
Canada uses a measure called the infiltration failure rate. This is the rate at 
which security screeners fail to detect “threat objects” such as simulated 
knives, simulated bombs, and simulated guns when inspectors attempt to 
carry them past the security screener. 


This information has been classified as secret and, accordingly, we may not 
report it. As Auditor General, I must respect the government’s information 
security regime. 


Parliament, however, faces a challenge in holding the government to account 
for security and intelligence activities. For example, passenger screening at 
airports involves weighing considerations such as how much security is 
desired, how long passengers are willing to wait, how intrusive a screening 
process they are prepared to accept, and what costs passengers and the rest of 
society are prepared to pay. While decisions on how to balance these 
questions are up to the government, it is up to Parliament to hold the 
government accountable for those decisions on behalf of Canadians. 


How can Parliament scrutinize the spending and performance of security and 
intelligence activities if key information must be kept secret? How will 
members of Parliament conduct an informed debate about security and 


intelligence matters? 
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One possible solution would be to create a mechanism such as a committee of 
Parliament, bound to secrecy and acting on behalf of the whole, that could 
receive reports containing classified information from security and 


intelligence agencies—perhaps in the form of secure annexes to their reports 
on plans and priorities and their departmental performance reports. 

This committee could also receive information from organizations such as the 
Security Intelligence Review Committee and the Office of the Auditor 
General, charged with scrutinizing these agencies’ activities on behalf 

of Parliament. 


Indeed, we understand that the government is contemplating the creation of 
such a mechanism, modeled somewhat on the United Kingdom’s practice. 
We encourage the government to move expeditiously on this. Security and 
intelligence have become a significant part of the federal government's 
activities and should receive the informed attention of Parliament. 


As it happens, four of the chapters in this Report cover government activities 
that have implications for national security. Our audits of the 2001 
anti-terrorism initiative, the Passport Office, and Natural Resources Canada 
looked at issues of national security or emergency preparedness. National 
Defence’s C4ISR initiative—which aims to use new information and 
communication technologies to improve military decision-making—has 
implications for national security as well, although our audit focussed on how 
the initiative was managed. 


Our chapter on the government’s 2001 anti-terrorism initiative focusses on 
emergency preparedness programs, the security of air transportation, and 
some elements of marine security. We reported on other aspects of national 
security in March 2004. 


Canada’s ability to respond to civil emergencies and acts of terrorism depends 
on the ability of federal, provincial, and municipal governments to effectively 
co-ordinate their procedures, their equipment purchases, and their training. 
We found that emergency response strategies are not integrated and are 
poorly co-ordinated. 


In many cases, spending on emergency preparedness was not guided by a 
thorough analysis of threats and risks; as a result, funds to strengthen 
emergency response capacities have been poorly allocated. For example, the 
opportunity was not taken to create a national pool of equipment that is 
compatible and interoperable. And despite the need for 6,000 people trained 
to respond to chemical, biological, radiological, and nuclear events, fewer 
than 200 have been trained so far. 


A new department, Public Safety and Emergency Preparedness Canada 
(PSEPC), has replaced the Office of Critical Infrastructure Protection and 
Emergency Preparedness. Without the needed authorities, structures, and 
resources, without strong and clear support from all areas of the federal 
government, PSEPC will be years away from meeting the goals the 
government has set for it. Meanwhile, the gaps in Canada’s ability to respond 
to emergencies will remain. 
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After the tragic events of September 11, 2001, the government created the 
Canadian Air Transport Security Authority to manage aspects of air travel 
security that needed to be radically expanded. 


Some key elements of air transport security are moving ahead well—for 
example, installing the explosives detection systems that screen passengers 
and baggage, which has cost some $1 billion. Other elements, however, are 
working less well. Transport Canada has expanded its security inspection 
service and improved training, but without a documented risk analysis we 
could not determine whether the number of inspectors and the frequency of 
inspections are adequate. 


In marine security programs, we found that spending has been based on an 
adequate threat and risk analysis and funds have been allocated to 
high-priority areas. However, there are still gaps in marine security— 

for example, in the coverage provided by the radar system designed for 
continuous surveillance of Canada’s coastal areas. How the gaps in coverage 
will be filled, and at what additional cost, has yet to be determined. 


To travel with ease across international borders, Canadians need an 

official travel document that is secure and universally respected. Since 
September 11, 2001, the Passport Office, responsible for issuing the 24-page 
blue document that Canadians know so well, has faced new demands and 
heightened expectations for passport security. Our audit found several areas 
that need urgent attention. 


In our view, significant improvements are necessary in the processes for 
determining whether documents submitted with an application are authentic 
and entitle the applicant to receive a passport. Passport “watch lists” are 
incomplete and often not updated in a timely way, because the Passport 
Office has not found ways to obtain data automatically from other 
government sources. Management's monitoring of some security functions is 
not sufficient to ensure that they are carried out properly. 


The Passport Office is struggling to find a balance between ensuring security 
and providing good service to clients. While it has established reasonable 
levels of service, its service standards do not take key security measures 

into account. 


In 2001, the cost of a passport rose from $60 to $85, but the increase did not 
comply fully with Treasury Board policy. The Passport Office is not ready to 
implement the requirements of new user-fee legislation. It lacks reliable cost 
information, has not engaged clients in establishing service standards, and 
does not have a mechanism to resolve fee disputes. 


At the time of our audit, the Passport Office did not have in place the 
management systems and practices it will need to meet future challenges. 
It needs to undertake a comprehensive risk assessment of all its operations, 
prepare an action plan, and closely monitor the plan’s implementation. 
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A 


Natural Resources Canada 


Resource industries in Canada employ more than one million Canadians 
and constitute over half the economic base of some 600 rural and remote 
communities across the country. Clearly, Natural Resources Canada has an 
important role to play in advancing government priorities. At the same time, 
the Department must fulfill its statutory responsibilities. 


To ensure that its departmental resources are allocated to critical priorities, 
Natural Resources Canada needs a corporate strategic plan that clearly sets 
out how it will achieve its goals. The plan must consider external threats and 
opportunities, available resources, and organizational strengths and 
weaknesses; it must spell out objectives, priorities, and expected results. 

The Department also needs a coherent approach to developing business plans 
for each of its sectors. Good governance and management processes are 
necessary at the corporate level to focus the Department’s efforts. 


Public safety is a strategic priority for all sectors of Natural Resources 
Canada 
The Department acts as the federal lead in planning and co-ordinating the 

federal response to civil emergencies related to natural resources; this would 


forests, earth sciences, energy, and mineral and metal resources. 


include major power shortages or failures, forest fires, and mine disasters. 


When we looked at the management of emergency preparedness across 
the Department, we found much room for improvement. The Department 
needs to use a more consistent approach to assessing risks across all sectors 
and ensure that appropriate emergency plans are established in all its areas 
of responsibility. 


National Defence has undertaken a major initiative to improve the way it 
gathers, processes, and uses military information for command and control. 
It expects that by 2015 it will have invested close to $10 billion in the C4ISR 
initiative—Command, Control, Communications, Computers, Intelligence, 
Surveillance, and Reconnaissance—to help make this transformation in 
information technology happen. 


Making command and control faster and more effective by providing 
better information for decision making is one goal of the initiative. Playing 
a meaningful role in operations with other countries such as NORAD and 
NATO partners means keeping pace with new military concepts 

and technology. 


C4ISR is a complex and expensive initiative; our audit found that National 
Defence has made a good start toward implementing it. However, with 

$4 billion already spent on C4ISR projects, some elements that are key to 
success—such as common standards and definitions 


are still not in place. 


and 


Without them, the result could be duplicate or incompatible systems 
the excess costs this would imply. 


National Defence recognizes that it needs to pursue C4ISR as a joint 
endeavour of its three services—Navy, Army, and Air Force—in order to take 
advantage of economies of scale and to achieve integrated systems that are fully 
interoperable. However, our audit found that the Department must do more to 
ensure that efforts are co-ordinated and that joint expectations are met. 
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Another chapter in this Report examines the role played by the federal 
government—in particular, Indian and Northern Affairs Canada—in the 
development of non-renewable resources in the Northwest Territories. 


With the signing of land claims agreements in the Northwest Territories and 
the passage of the Mackenzie Valley Resource Management Act seven years ago, 
the federal government created a series of boards to manage the application 
and licensing process for resource development and to regulate land and 
water use for environmental protection. At the time, Indian and Northern 
Affairs Canada decided that the best course of action was to leave the boards 
to administer the process on their own. It is now time to re-examine 

this approach. 


The Department has authority under the Act to provide certain direction to 
the boards, authority it has chosen not to exercise. Providing direction would 
ensure that necessary elements are in place for the smooth functioning of the 
process. These would include standards for water quality and guidelines for 
interpreting key terms in the legislation. 


The continued absence of policy direction, the lack of appropriate 
accountability, and inadequate management have contributed to the 
uncertain and unstable climate for investment in non-renewable resources 
that hampers economic development in the Northwest Territories. 


Canadians want to know the value they are getting for their tax dollars and 
the difference government is making in their lives. To monitor government 
programs and services effectively, parliamentarians need to be provided with 
timely and accurate information about what government spends and what 
results it achieves. Good performance reports are a key means for both 
Parliament and Canadians to hold the government accountable. 


Every fall, some 90 federal departments and agencies submit reports to 
Parliament on their performance. These reports outline departmental 
accomplishments at the end of the fiscal year against commitments made in 
the previous year’s plans. 


For some time, Parliament has sought to improve the quality of the 
information in these performance reports. And my Office has supported 
Parliament’s efforts. In 1997 we examined the state of the government’s 
reporting regime and found a basic but sound framework as a promising start 
for reporting. But our follow-up audit in 2000 concluded that little progress 
had been made in improving the quality of performance reports produced 

by departments. 


In 2002 we presented a model for rating performance reports. We then 
applied our rating model to the reports of nine departments, and in 2003 we 
reported the results. While we identified some promising practices, overall we 
found that most reports did not measure up well against our model. 


This is the second time we have used our model to rate a selection of 
performance reports—those of Environment Canada, Fisheries and Oceans 
Canada, and Natural Resources Canada, who were part of a project in 1995 
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to improve reporting. We looked at their progress over the last decade and 
then compared it with their progress since 2002. The results were 


disappointing. 


Without greater effort by departments and greater scrutiny by parliamentary 
committees, it will likely take not years but decades to significantly improve 
the quality of performance reports. In my view, this rate of improvement is 
not good enough to give Parliament and Canadians the tools for effectively 
holding government departments and agencies accountable for 

their performance. 
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Natural Resources Canada 
Governance and Strategic Management 


Main Points 


1.1 Natural Resources Canada has been working on a number of 
significant issues. However, the Department does not have a corporate 
strategic plan that addresses its legislative mandate and government 
priorities, is communicated to staff, and serves to align sector business plans. 
It needs good governance and management processes at the corporate level 
to focus it efforts. It also needs to improve its strategic decision-making and 
governance processes to help ensure that horizontal issues are managed 
consistently across the Department. 


1.2 The Department is the federal lead in developing civil emergency plans 
for co-ordinating the federal response to emergencies in a number of areas 
related to natural resources. We found that the Department’s operating 
sectors did not use a coherent framework for assessing risk and that the 
Department does not have appropriate emergency plans in all of its 
responsibility areas. 


Background and other observations 


13. The Department is faced with an aging, specialized workforce. Yet, it 
does not have a clear understanding of the competencies and capacities of its 
current workforce and those that it will need to acquire. We also found that 
the Department’s information management systems lack consistent data that 
would allow effective corporate oversight. 


14 Natural Resources Canada generally meets its statutory obligations 
with respect to its reporting requirements under the Report on the State of 
Canada’s Forests Regulations and the Energy Efficiency Act. The Department 
needs to establish a more rigorous process that could help it move toward 
managing for results. Its performance reports need to address the 
Department’s mandate and make its various frameworks for assessing 
performance easier to understand. 


15 The Department has a very broad mandate, and it faces a number of 
significant challenges. These include funding levels that were cut by about 
50 percent and then doubled over the last 10 years. Almost half of its funding 
is for short-term programs that expire in three to five years. In addition, many 
of its executives and technical specialists are eligible to retire. Recently, the 
Department has undertaken a number of important initiatives to deal with 
the issues it faces. It is important for the Department to have good strategic 
management systems and practices to ensure that it coherently deals with 


increasingly horizontal issues. 
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The Department has responded. Natural Resources Canada has indicated 
the actions it has under way or has planned to address the recommendations. 
Its detailed response follows each recommendation throughout the chapter. 


2 
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Chapter 2 


National Security in Canada— 
The 2001 Anti-Terrorism Initiative 


Air Transportation Security, Marine 
Security, and Emergency Preparedness 


Main Points 


21 In December 2003—two years after it launched its anti-terrorism 
initiative—the government reorganized federal emergency planning, creating 
Public Safety and Emergency Preparedness Canada. The new department 
began an extensive reform program, but much remains to be done. Threat 
and risk analyses were not always used to guide the disbursement of funds by 
the previous agency to municipalities and provinces, resulting in poor 
allocation of funds. We found in a number of cases that programs do not 
appear to have achieved the intended results: 


* In 2002 to 2004, federal funds were used to purchase equipment for first 
responders that did not have to be interoperable, losing a key 
opportunity to enhance the national capacity to respond to an 
emergency. 


* The Office of Critical Infrastructure Protection and Emergency 
Preparedness, responsible for implementing a critical infrastructure 
protection initiative of Budget 2001, could not tell us what it had spent 
funds on. About one third of the funds allocated so far to this program 
may have lapsed. 


2.2 Governance structures require continued attention. The government 
has established the Government Operations Centre to manage emergency 
responses, but the legislative and infrastructure frameworks remain 
uncompleted. Managing the budgets of projects that cut across several 
departments worked well in some cases, such as marine security and the 
CBRN Research and Technology Initiative, but failed in the cases of CBRN 
training for first responders and critical infrastructure protection. (CBRN 
refers to chemical, biological, radiological, or nuclear threats.) 


23 The departments responsible for the CBRN Training Initiative 
estimated that about 6,000 first responders should be trained in how to 
intervene in and neutralize a serious event. We found that 200 people had 
been trained to this level so far under the CBRN Training Initiative. 


24 Some key elements of the federal government’s air transport security 
program are moving forward. The purchase and installation of explosives 


detection systems to screen passengers and baggage—an initiative by the 
Canadian Air Transport Security Authority (CATSA) at a cost of over 
$1 billion—are proceeding mostly as planned. Analyses of requirements and 


options were generally adequate. 
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25 Other elements, however, are working less well. Transport Canada told 
us it does not have a major problem with CATSA, but the Department has 
set no system-wide performance levels for CATSA and completed no 
assessments of its performance. The Department has not defined the 
sanctions it will use against CATSA if education and persuasion fail to get 
timely results. 


2 Marine security programs have been based on an adequate threat and 
risk analysis. Implementing the Automated Identification System for ships 
encountered delays but is still expected to be completed by its original 
implementation date. The High Frequency Surface Wave Radar project, 
intended to provide continuous, real-time surveillance of Canada’s coastal 
areas, represents an improvement in surveillance but will not provide full 
coverage. The government is acquiring fewer radar sets than considered 
necessary for complete coverage, and the radar will have a more limited range 
than originally forecast. The result will be gaps in coverage, with significant 
additional costs if full coverage is to be attained. 


27. The government's implementation of the International Ship and Port 
Facility Security Code, necessary to keep Canadian ports and ships in the 
business of world trade, is generally going well. However, some certificates 
were issued on an interim basis, and security inspectors may be pressed to 
complete the inspections required for new certificates by the renewal 
deadline. 


Background and other observations 


28 This is the second of two audit reports on the federal government's 
anti-terrorism initiative announced in the 2001 Budget after the 
September 11 attacks on the United States. The government initially 
announced $7.7 billion for this initiative and subsequently added another 


$690 million. 


29 Our first report, published in March 2004, addressed budget 
management, the allocation of funds, monitoring of expenditures, 
intelligence issues, and some aspects of air and border security. This chapter 
addresses air transportation security, elements of marine security, and 
emergency preparedness programs. 


The departments and agencies have responded. In general, the 
organizations have agreed with our recommendations, although 
commitments toward remedial action remain vague in some cases. We found 
the response from Public Safety and Emergency Preparedness Canada to be 
positive, as the Department is already moving ahead in some areas. 
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Passport Services 


Main Points 


3.1 The Passport Office is struggling to meet increasing security 
expectations and demands for service. Significant improvements are 
necessary in the processes for determining passport entitlement. The Office is 
currently unable to fulfill all of its responsibilities under the Canadian 
Passport Order. Its watch list is deficient and often not updated in a timely 
fashion because the Office has not found ways to automatically obtain data 
from other government sources. Management does not sufficiently monitor 
some key security functions to ensure that they are properly carried out. 


3.2 The Passport Office has established and achieved its published service 
standards for clients. However, costs are increasing and are not tied to service 
standards. Although it has reported on the performance of certain major 
service standards, there is limited performance information about security 
measures. At the time of the audit, the Passport Office did not have the 
management systems and practices that would prepare it to meet future 
challenges effectively. 


33. The Passport Office and Foreign Affairs Canada did not meet the 
requirements for ongoing consultation on service standards and how they 
relate to passport and consular fees. The Passport Office also lacks reliable 
cost information related to published service standards. 


Background and other observations 


34 The Passport Office is a Special Operating Agency reporting to Foreign 
Affairs Canada. It finances its operations from the user fees it collects. The 
Office issued about 2.5 million passports in 2003-04. It collected revenues of 
$158 million that it used to finance its operations and had a net income of 
$9 million. It also collected $54 million in consular fees for deposit to the 
Consolidated Revenue Fund. The Passport Office is responsible for enforcing 
the Canadian Passport Order in issuing various travel documents. Issuing the 
24-page blue passport, which in 2005 rose in cost from $85 to $87, makes up 
about 98 percent of Passport Office business. 


The Passport Office has responded. In its response to each recommendation 
throughout the chapter, the Passport Office indicates the action it has taken, 


is taking, or plans to take to address the recommendation. 
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C4ISR Initiative in Support of 
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41 National Defence estimates that by 2015 it will have invested almost 
$10 billion on projects to improve the way it gathers, processes, and uses 
military information. This is needed to provide commanders with better 
information for decision making in order to exercise faster and more effective 
command and control in both joint and combined operations. It is also to 
allow National Defence to keep up with the progress and changes being made 
by allies. This is a key part of the transformation of the Canadian Forces into 
the 21st century. The Command, Control, Communications, Computers 
(C4), Intelligence, Surveillance, and Reconnaissance (ISR) initiative is a 
strategy to help make this information-technology transformation happen. 


42 The Department has made a good start in managing C4ISR and has 
put in place some guidance and methodology to help it achieve its goals. 
However, some of the key elements required to ensure successful 
implementation of the C4ISR initiative are not yet in place. National 
Defence must put a priority on producing its joint C4ISR doctrine, a concept 
of operations, a clear definition of interoperability, and a common 
understanding of what C4ISR means to better guide its development. 
National Defence has already invested about $4 billion of the almost 

$10 billion planned for projects that have a C4ISR component; however, 
without these key elements the Department is at risk of developing non- 
compatible or duplicate systems. 


43 C4ISR is complex and expensive and constitutes a significant 
investment by the Department in improving the way it collects, processes, 
and uses information for operations, in particular joint and combined 
operations. The Department recognizes that the individual services cannot 
afford to pursue, nor should they pursue, C4ISR separately. The Navy, Army, 
and Air Force must work jointly to take advantage of economies of scale and 
achieve fully interoperable and integrated systems. While the Deputy Chief of 
Defence Staff is responsible for joint operations, the individual services have 
traditionally been responsible for generating what they need for those 
operations; therefore, C4ISR-related development has tended to be pursued 
along individual service paths rather than jointly. As well, existing joint 
requirements committees need clearer authority and a stronger role in project 
approval to ensure that activities co-ordinate and meet joint expectations. 


44 ‘To have a C4ISR initiative that is affordable and achievable, National 
Defence needs mechanisms that help it choose which projects to pursue and 
to guide how systems will be developed. The Department is using a 


Report of the Auditor General of Canada—April 2005 


MAIN POINTS 


methodology that helps it define and refine C4ISR needs over time and has 
begun working on a common approach to designing its systems to meet those 
needs. However, projects already under way or being developed have not 
followed a common design approach and need to be reviewed to ensure they 
still meet the Department’s intent. Without this adherence to a common 
approach, the Department has no assurance that its systems will converge 
and integrate as planned. Both a common design approach—or enterprise 
architecture—and a review of all projects need to be completed if the 
Department is to move forward in a structured, efficient, and disciplined way. 


Background and other observations 


45 The goal for the Canadian Forces, like other militaries around the 
world, is to improve the way it collects, analyzes, disseminates, and shares 
information gathered through C4ISR means to provide commanders with 
trusted and relevant information for decision making. Militaries are putting 
renewed efforts into C4ISR because of the opportunities information 
technology provides to enhance their own command and control. 


46 In December 2003, the Department of National Defence released its 
C4ISR Command Guidance and Campaign Plan document to provide the 
Canadian Forces with high-level guidance and an integrated approach to 
develop and transform the capability into one that supports Forces-wide 
command and control into the 21st century. 


47 By 2008, the Department wants to achieve its first C4ISR timeline and 
to have completed three overall phases—concept development, . 
consolidation of projects and initiatives, and initial transformation; these 
involve the creation of an information-based culture and a network-enabled 
organization. 


48 Our 1994 Report, Chapter 25, Information Technology recommended 
that command and control systems be able to interoperate in joint and 
combined operations, which the Department accepted as a mandatory 
requirement. The Information Technology Infrastructure and the Canadian 
Forces Command and Control Information System were both actively 
addressing these requirements. The Department acknowledged that joint 
interoperability had not received a high priority in these activities until the 
recent past. Nevertheless, our 1996 follow-up chapter concluded that the 
interoperability of command and control communication systems needed 
further development for joint and combined operations. 


49 In our 1998 Report, Chapter 3, Equipping and Modernizing the 
Canadian Forces, we reported that, as far back as 1994, the Army had not 
kept pace with technology to modernize its equipment; this left it vulnerable 
to threats in low-level and mid-level operations. Some of those problems have 
since been corrected, but some have not. For example, National Defence has 
been working on implementing an Army communications system, which was 
to be completed by 2001 and was to be fully interoperable with other 
Canadian Forces command and control systems. By 2004, however, this was 


still not fully operational. 
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The Department has responded. The Department has indicated the action 
it has taken or intends to take to address the recommendations. Its detailed 
response follows each recommendation throughout the chapter. 
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Rating Selected Departmental 
Performance Reports 


Main Points 


5.1. We assessed the quality of the departmental performance reports of 
three departments—Environment Canada, Fisheries and Oceans Canada, 
and Natural Resources Canada—to determine whether progress had been 
made in improving the quality of their reports against our criteria. In assessing 
the departments’ performance reports against our rating model, we found that 
the quality of the reports had improved marginally, at most, over the two-year 
period between 2002-03 and 2003-04. Over the nine-year period, between 
1995-96 and 2003-04, we found that two departments had achieved some 
modest improvements while the other department showed mixed results. 
Despite these modest improvements in performance reporting over the nine 
years, the latest performance reports still fall short of our criteria for good- 
quality reporting. 


5.2 We found that most of the performance reports provide a good 
overview of each department’s organizational context and planned strategic 
outcomes. However, performance expectations are not always clear and 
concrete, and the information does not necessarily focus on program results. 
Furthermore, the reported results are not always balanced and supported by 
data sources and data limitations. As well, the reports provide little evidence 
that performance information is used to make decisions about improving 
program results in future years. 


5.3. While we cannot generalize from the results of rating three reports in 
each of three years to trends across all government departments, our findings 
continue to leave us as concerned about the overall quality of reporting as we 
were in our previous audits and studies. Based on these findings, it is 
reasonable to suggest that, without greater effort by departments and scrutiny 
by parliamentary committees, any significant improvements in the quality of 
performance reporting may take decades rather than just a few years. In our 
view, this rate of improvement is not good enough for parliamentarians and 
Canadians to be able to hold departments and agencies to account for their 
performance. 


Background and other observations 


5.4 Parliament holds government accountable for spending in the previous 
year, influences future plans and priorities, and approves expenditures for the 
upcoming fiscal year. Parliamentarians should rely on performance reports to 
keep informed about the performance of the government on key issues. They 
should also use these reports to hold ministers to account for departmental 
spending. As well, Canadians want to know if their taxes were spent where 
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parliamentarians intended. They also want to know if they received good 
value from their government in return for their taxes. Consequently, effective 
accountability to parliamentarians and Canadians depends in large part on 
good-quality performance reporting. 


5.5  Inresponse to this need, Parliament and the government have sought 
to improve the quality of departmental performance reporting for some time. 
For example, Part III of the Estimates was created in 1981 and then separated 
into planning and reporting documents on a pilot basis in 1995 and 
permanently in 1997. 


5.6 Our Office has also been concerned about the quality of performance 
reporting. We have reflected this concern through our audits since at 

least 1988 when we found clear shortcomings while also noting some 
improvements. In 2000 we examined the quality of departmental 
performance reports and concluded that federal departments’ and agencies’ 
progress in improving the quality of their performance reporting to Parliament 
was disappointing. 


§.7 To assist us in our subsequent work, we developed a formal set of 
criteria for good reporting and presented it in 2002 as a model for rating 
departmental performance reports. In the following year, we reported on the 
results of applying this rating model to the departmental performance reports 
of nine departments. While we identified some promising practices, together 
with many challenges, overall we found that the reports did not measure up 
against our model as well as we had expected. 


5.8 This is the second time that we have used our model to rate a selection 
of performance reports. However, it is the first time we have compared a 
department’s reports over one or more periods to assess progress in improving 
the quality of reporting. Overall, we found that progress in improving 
performance reporting to Parliament continues to be a challenge for the three 
departments, even after almost a decade of effort. 


The government has responded. The government has responded to our 
observations and informs us that it will continue its efforts to improve 
performance reporting across government departments. 


10 
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Indian and Northern Affairs Canada 


Development of Non-Renewable 
Resources in the Northwest Territories 


Main Points 


6.1 | Non-renewable resources offer enormous potential for economic 
development in the Northwest Territories (NWT). Yet the investment 
climate for this development is uncertain, in part because Indian and 
Northern Affairs Canada has not adequately managed its role in the process 
that considers development projects. 


6.2 This includes not providing guidance on some of the ambiguous terms 
in the governing legislation or on establishing water standards permitted by 
legislation. It also includes not requiring boards to be accountable for 
managing their role in the process without impinging on the decisions they 
take as quasi-judicial bodies. 


Background and other observations 


6.3 Our audit examined how well Indian and Northern Affairs Canada 
(INAC) has managed its responsibilities that form part of the application and 
licensing process for the development of non-renewable resources in the 
Northwest Territories. 


64 With the signing of land claim agreements in the Northwest Territories 
and the passage of the Mackenzie Valley Resource Management Act (MVRMA), 
the federal government created a series of boards to regulate the use of land 
and water and protect the environment. When these boards were created, the 
Department felt that the federal government’s best course of action was to 
leave the boards to administer the process on their own in order to ensure that 
the Aboriginal peoples of the NWT understood that the federal government 
was sincere in relinquishing control. That decision seven years ago has 
contributed to a regulatory environment that, today, needs strengthening. 


6.5 As the federal government continues negotiating the transfer of 
responsibilities to the Northwest Territories, it is important for Canadians 
that the process for developing non-renewable resources that the NWT may 
inherit is well managed. 


6.6 Furthermore, seven years since the passage of the Act, it is timely for 
the Department to re-examine its approach to managing the process and 
strengthen it in the areas described in this report to ensure that it is 
sufficiently robust to meet the challenges and realize the opportunities of the 
coming decade. 


The Department has responded. Indian and Northern Affairs Canada 
agrees with all the recommendations and has committed to taking action to 


address the concerns we raise in this chapter. 
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Natural Resources Canada 


Governance and Strategic Management 


All of the audit work in this chapter was conducted in accordance with the standards for assurance engagements set by the 
Canadian Institute of Chartered Accountants. While the Office adopts these standards as the minimum requirement for our audits, 
we also draw upon the standards and practices of other disciplines. 
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Governance and Strategic Management 


Main Points 


11 Natural Resources Canada has been working on a number of 
significant issues. However, the Department does not have a corporate 
strategic plan that addresses its legislative mandate and government 
priorities, is communicated to staff, and serves to align sector business plans. 
It needs good governance and management processes at the corporate level 
to focus it efforts. It also needs to improve its strategic decision-making and 
governance processes to help ensure that horizontal issues are managed 
consistently across the Department. 


12 The Department is the federal lead in developing civil emergency plans 
for co-ordinating the federal response to emergencies in a number of areas 
related to natural resources. We found that the Department’s operating 
sectors did not use a coherent framework for assessing risk and that the 
Department does not have appropriate emergency plans in all of its 
responsibility areas. 


Background and other observations 


13. The Department is faced with an aging, specialized workforce. Yet, it 
does not have a clear understanding of the competencies and capacities of its 
current workforce and those that it will need to acquire. We also found that 
the Department’s information management systems lack consistent data that 
would allow effective corporate oversight. 


14 Natural Resources Canada generally meets its statutory obligations 
with respect to its reporting requirements under the Report on the State of 
Canada’s Forests Regulations and the Energy Efficiency Act. The Department 
needs to establish a more rigorous process that could help it move toward 
managing for results. Its performance reports need to address the 
Department’s mandate and make its various frameworks for assessing 
performance easier to understand. 


15 The Department has a very broad mandate, and it faces a number of 
significant challenges. These include funding levels that were cut by about 
50 percent and then doubled over the last 10 years. Almost half of its funding 
is for short-term programs that expire in three to five years. In addition, many 
of its executives and technical specialists are eligible to retire. Recently, the 
Department has undertaken a number of important initiatives to deal with 
the issues it faces. It is important for the Department to have good strategic 
management systems and practices to ensure that it coherently deals with 
increasingly horizontal issues. 


Report of the Auditor General of Canada—April 2005 


Chapter 1 1 


NATURAL RESOURCES CANADA—GOVERNANCE AND STRATEGIC MANAGEMENT 


The Department has responded. Natural Resources Canada has indicated 
the actions it has under way or has planned to address the recommendations. 
Its detailed response follows each recommendation throughout the chapter. 
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Introduction 


\ 


16 Natural Resources Canada is a successor to the Department of Energy, 
Mines and Resources and an amalgam of previously separate and mainly 
science-based institutions, including the Canadian Forest Service, the 
Geological Survey of Canada, Geomatics Canada, and the Department of 
Mines. 


17 The Department is organized along industry lines and comprises five 
operational sectors and a corporate services sector (Exhibit 1.1). It is the 
federal lead on forestry, energy supply and use, minerals and metals, and earth 
sciences, and it works closely with partners and stakeholders. 


18 Canada’s resource industries are a vital part of the national economy, 
accounting for more than $131 billion and more than 13 percent of the gross 
domestic product in 2002. Over one million Canadians are directly employed 
in the resource industries across Canada. Over 600 rural and remote 
communities, out of a total of nearly 6,000 such communities across Canada, 
rely on the resource industries for more than 50 percent of their economic 


base (Exhibit 1.2). 


Exhibit 1.1 Natural Resources Canada organization chart 


Parliamentary Secretary 
Deputy Minister 
Special Advisor 
Canadian Forest Service 
Corporate Services Sector 
Earth Sciences Sector - 
Energy Policy Sector 
Energy Technology and j 
Programs Sector 
Minerals and Metals Sector } 


Source: Natural Resources Canada 
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Exhibit 1.2 Communities that rely on forestry, energy, mining, and metal fabrication, 2001 


Communities 
Resource-reliant communities (approximate) 
Forestry 300 
: 
Energy 75 
Mining and metal fabrication 125 
4 
Communities that rely on a combination of the above-noted 165 
resource industries 


Total 665 


Source: Natural Resources Canada 


19 The Department faces a number of major horizontal, international, 
economic, and policy issues that can be broken down into three categories: 
supply issues, market issues, and environmental issues (Exhibit 1.3). 
Horizontal issues are those that affect more than one sector or federal 
department. 


Exhibit 1.3 Major departmental issues 


Supply issues 


¢ Offshore oil and oil sands development 
¢ Northern energy and pipeline development 


¢ Aging electricity supply infrastructure (including nuclear energy plants in Ontario, 
Quebec, and New Brunswick) 


¢ Depleting reserves of seven major Canadian metals (for example, copper and zinc) 
* Energy infrastructure protection 
¢ Alternative energy and demand issues, including energy efficiency 


Market issues 


¢ Regulatory harmonization 

* Softwood lumber and U.S. market access 
* International market access 

* Resource taxation 


Environmental issues 


* Climate change 
* New, foreign, invasive forest pests 


Source: Natural Resources Canada 
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Fluctuations in the Department’s scientific workforce 


110 Asa result of the government’s program review cuts, the Department’s 
expenditures were reduced by about 50 percent between 1995 and 1998, and 
its staff was reduced by about 30 percent. Reductions were distributed mainly 
among scientific (794) employees, management (79), and administrative 


(450) categories (Exhibit 1.4). 


Exhibit 1.4 Departmental changes in expenditures and staffing 


Expenditures ($ thousands) 


1,135 
1,030 
eae 
517 
1993-94 1997-98 2004-05 
actua actual estimate 
Full-time equivalent staff 
5,426 
4,356 
SHAM 
1993-94 1997-98 2004-05 
actual actual estimate 


Research scientists and managers 


68 


| Research managers 


i Research scientists 


1993-94 1997-98 2004-05 
actual actua estimate 


Source: Natural Resources Canada 
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1.11 As of September 2004, about 70 percent of the Department's staff are 
engaged in science and related scientific activities, including 498 research 
scientists and 74 research managers. The Department spends about 

74 percent of its budget (excluding statutory payments) on science and 
related activities. (Statutory payments of about $334 million comprised 
mainly transfers to provinces for offshore oil and gas development and 
contributions to employee benefit plans.) Following the government's 
program review, the Department focussed on its remaining in-house science 
activities, while maintaining some policy capacity and regulatory oversight. 


1.12 The knowledge and expertise of Natural Resources Canada’s workforce 
are vital to the effectiveness of its programming. Similar to many other 
knowledge-based organizations, the Department must confront the problem 
of an aging workforce. 


Reductions in base funding and increases in sunset funding 


1.13 The Department’s resources have increased from the lower levels 
following program review. However, the increase in staff has been mainly the 
result of the introduction of temporary funding for special purpose and sunset 
programs. These programs have a short life-span, generally from one to five 
years. Natural Resources Canada estimates that these programs now comprise 
62 percent of the Department’s budget. Sunset programs have a number of 
impacts, including the following: 


+ Management attention is drawn from other work to manage the 
changing resources. 


* It is often difficult or impossible to move scientific staff from one project 
to another in an unrelated field. 


* Scientists are more difficult to recruit as they prefer to work for 
organizations with more assured funding. 


Recent initiatives 


1.14 In response to the challenges it faces, the Department has undertaken 
a number of important initiatives in the past two years. 


* Support services. The Department completed a major study of support 
services (finance, procurement, information technology, 
communications, asset management, and human resources) and is in the 
process of implementing its recommendations. This involves the 
creation of a Shared Services Office to improve the effectiveness and 
efficiency of support services and to strengthen corporate administrative 
management. 


* Science management. The Department created the Office of Chief 
Scientist in September 2003. It informed us that the Office is currently 
developing a vision document and an organizing architecture that will 
include governance structures, principles, guidelines, and management 
policies and processes for science and technology. The Department 
indicated that it is also considering theme-based networks as a means to 
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improve co-ordination of science and technology programs that address 
cross-cutting policy issues. 
‘ 


* Information management. The Department informed us that it has 
developed options for a departmental science and technology 
information system. Development of this system has been put on hold 
due to a Treasury Board decision on the potential development of a 
government-wide information system. In the meantime, the Department 
is developing a standardized template for the operational sectors to 
report science and technology information. 


* Human resources management. Initiatives in the past year include the 
Policy Analysis Recruitment and Development Program. Through this 
program, the Department recruited university graduates to enhance its 
policy capacity. 


The Department’s legislation is broad 


1.15 The Department’s enabling legislation is set out in over 30 acts of 
Parliament. This legislative base has developed over more than 100 years. 
The four core acts are the Department of Natural Resources Act, Resources and 
Technical Surveys Act, Explosives Act, and Forestry Act. 


1.16 The Department of Natural Resources Act sets out the main purposes of 
the Department. It requires the Minister of Natural Resources to 


* co-ordinate, promote, recommend, and implement policies; 


* assist in the development of Canadian scientific and technological 
capabilities; 


* promote the development and use of remote sensing; 
* develop codes and standards for surveys and natural resource products; 
* gather, compile, analyze, co-ordinate and disseminate information; 


* seek to enhance the responsible development and use of Canada’s 
natural resources and the competitiveness of Canada’s natural resources 


products; 
* participate in the enhancement and promotion of market access; 


- have regard for the sustainable development of Canada’s natural 
resources and the integrated management thereof; and 


* promote co-operation with governments and non-governmental 


organizations in Canada and internationally. 


117. The remaining acts set out terms for energy policy; nuclear policy; 
management of Crown lands and boundaries; and regulation of the sale, use, 
and distribution of explosives; they also establish a foundation to fund 


sustainable development technology. 
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The Department has interpreted its role in different ways 


118 In response to changing government priorities, the Department has 
focussed on different areas over time and interpreted its role as one or more of 


the following: 


* knowledge provider, especially for the materials that compose Canada’s 
earth; 


* manager and promoter of science and technology; 
* regulator; 


* manager of developmental programs in the energy, forest, minerals, and 
metals sectors; 


* investor in mega-energy projects; and 


* promoter of energy efficiency. 


1.19 Natural Resources Canada is responsible for developing and 
maintaining emergency plans related to a number of different areas. These 
include co-ordinating the federal response to energy shortages or major power 
failures (for example, the power outage of 14 August 2003 that affected 
Ontario and the Eastern United States), mine disasters, and emergencies 
associated with offshore oil and gas production. 


1.20 The Department is jointly responsible with the National Energy Board 
for the federal role in the control and regulation of energy production, 
generation, processing, transmission, storage, sale, domestic distribution, and 
exports and imports. Indian and Northern Affairs Canada is responsible for 
the federal role in this area in the territories. 


How Natural Resources Canada views its operations 


1.21. Natural Resources Canada has the mandate to promote the sustainable 
development and use of Canada’s mineral, energy, and forestry resources and 
to enhance the competitiveness of Canada’s resources and products. 


1.22. The Department’s activities are extremely varied across its business 
lines. In particular, it deals extensively with three major sectors of the 
Canadian economy: energy, forestry, and mining. While there are common 
horizontal policy themes (for example, sustainability issues, regulatory issues, 
and climate change implications), the issues, opportunities, and challenges 
facing the three sectors are quite different from the perspective of 


* commodity price cycles and markets associated with the sectors, 
¢ their resource bases, 
* the international orientation of the three sectors, and 


+ their stakeholder groups. 
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123 The Department works closely with industry and stakeholder groups 
associated with the three commodity sectors. Some of the specific initiatives 
that the commodity sectors have advanced include the following: 


* Canadian Forest Service. The establishment of the Canadian Forest 
Innovation Council (with representation from industry and the 
provincial governments) and greater collaboration with provincial and 
territorial forestry ministers are intended to lead to more strategic and 
co-ordinated approaches to the forestry agenda. 


* Energy Policy Sector and Energy Technology and Programs Sector. 
As part of the Energy Dialogue Mechanism, the Department is working 
closely with provincial officials and industry representatives to provide 
leadership on energy policy issues. 


* Minerals and Metals Sector. The Kimberley Process brought together 
the governments of 50 countries, the diamond industry, and non- 
governmental organizations to curb the trade in conflict diamonds. 


1.24 The fourth sector in Natural Resources Canada—the Earth Sciences 


Sector—provides unique and essential components of the science required to 


make informed economic, environmental, and social policy decisions for the 
current and future governance and development of Canada. In essence, the 
Sector serves as a science and information-based service provider to 
government and other stakeholders. 


1.25 The Department’s key planning challenge is to integrate the 
management of horizontal issues with sectoral strategies. Examples of 
horizontal policy issues that have received focus across the Department 
include the following: 


* a framework for sustainable development; 
¢ climate change mitigation and adaptation; 
- regulatory affairs; 

* innovation; and 


* science and technology management. 


1.26 Climate change has been a major focus of activity and new programs 
within the Department. Natural Resources Canada’s role has been led by the 
Energy Policy Sector but has involved all sectors; the Department has also 
been part of a much larger government-wide effort, including shared planning 


and accountability systems. 


1.27. The Department is supported in its horizontal operations by corporate 
units (including the Corporate Policy and Portfolio Coordination Branch, the 
Office of the Chief Scientist, and the Communications Branch) as well as 
horizontal management committees (for example, the Departmental 
Management Committee and the Strategic Policy Committee). 


1.28 The Department’s organization, planning activities, and management 
processes at the corporate level are structured to balance the Department's 


sector-specific responsibilities and activities with the need for sound 
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management of horizontal policy issues. These processes include the 


following: 


« Natural Resources Canada’s corporate branches carry out broad 
planning and contextual strategic policy analyses that serve all the 


sectors. 


¢ Each sector develops a tailored approach to strategic sectoral 
management and operations, consistent with its operational 
responsibilities and in keeping with the specific policy challenges and 
opportunities facing the sector. 


¢ Horizontal departmental operations (including support services, science 
management, and human resource management) are also managed at 
the corporate level. 


¢ Horizontal government-wide priorities (including climate change, the 
development of a sustainable development framework, regulatory 
reform, and water issues) are supported by both the sectors and the 
corporate branches. 


Focus of the audit 


1.29 Our audit objectives were 


* to assess whether Natural Resources Canada’s governance mechanisms, 
management systems, and practices at the strategic level are adequate to 
support its Department-wide policy development, research, program, 
and other activities; and 


* to determine whether the Department appropriately assesses and reports 
to Parliament the results and impacts of its efforts and meets its statutory 
reporting obligations. 


1.30 Our audit focussed on the governance and management practices of 
Natural Resources Canada’s five operational sectors at the strategic level 
rather than the program level. Thus, we did not audit the Department’s 
programs such as climate change; nor did we audit interdepartmental 
initiatives. In order to review strategic management across the Department, 
we selected emergency preparedness for further testing. This activity is part of 
public safety and security, one of the Department's five strategic outcomes. 


1.31 We identified four lines of enquiry: 
* strategic management and direction setting; 
* integrated information management and decision making; 
* aspects of human resources management; and 


* performance measurement and reporting. 


1.32. Some quantitative information in this chapter is based on data 
provided by Natural Resources Canada. We assessed these data through a 
process of analysis, comparison, and discussion. Unless otherwise indicated in 
this chapter, data provided by Natural Resources Canada should be treated as 
unaudited. More details on the audit’s objective, scope, approach, and criteria 
are included in About the Audit at the end of the chapter. 
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Observations and Recommendations 


iy 


133 Each of the Department's five operating sectors has unique 
characteristics that include legislative responsibilities, stakeholders, and key 
issues. We would not expect one overall integrated business plan to serve the 
needs of all sectors. However, we would expect the Department to have a 
corporate strategic plan to provide guidance to the sectors and to clearly 
explain the Department’s overall strategic priorities to parliamentarians and 
stakeholders. 


1.34 The Department faces a number of challenges, including 
* asevere workforce reduction followed by a rapid increase, 
* an aging workforce, and 


* a significant amount of its resources dedicated to sunset programs that 
make staffing more difficult. 


In these circumstances, it is important that the Department have good 
strategic planning to help it allocate its scarce resources and adapt to its 
changing environment (Exhibit 1.5). 


Exhibit 1.5 Elements of a good strategic plan 


A good strategic plan 


* is consistent with mandate and government priorities; 


¢ considers factors such as external environment, risks, options, stakeholders, 
available resources, organizational strengths and weaknesses, and potential impacts; 


* analyzes risks, options, and proposed courses of action; 

¢ sets out organizational vision, priorities, objectives, and expected results; 
* is communicated to staff; and 

¢ is implemented throughout the organization. 


Strategic planning documents are inconsistent 


1.35 Natural Resources Canada does not have a corporate strategic plan. 
Instead, it uses two classified documents, a 2004 briefing document and a 
Vision Paper, to identify its organizational vision, priorities, and direction. 
Members of the Department’s management committee, including the heads 
of the operational sectors, helped develop these documents. However, since 
these documents are not available to staff and other stakeholders, we looked 
at other authoritative documents—the sustainable development strategies, 
sector business plans, and the reports on plans and priorities—to see if they 
set out strategies consistent with the classified documents and communicated 


these strategies to staff and other stakeholders. 


136 The Commissioner of the Environment and Sustainable Development 
noted in October 2004 that Natural Resources Canada was among the 


departments that have often performed relatively well in senior leadership, 
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with benefits including better sustainable development strategies and more 
decisive efforts to deliver on environmental commitments. 


1.37 Sustainable development strategies are intended to influence change 
toward sustainable development. The Department’s 2004 strategy was not 

intended to cover all areas of its responsibilities; therefore, in our view, the 

strategy cannot fulfill the role of a corporate strategic plan. 


1.38 We found that each of the five operational sectors is responsible for 
establishing its own mission and ensuring it is consistent with relevant 
legislation and government direction. While this is necessary, the 
Department has no coherent approach for analysis across the sectors to 
ensure that they adequately address their legislative mandates or the priorities 
of Parliament, and to assist in the management of horizontal issues. As a 
result, sector mission statements did not state the results expected in 
coherent terms, with clear links to the Department’s vision, legislated 
mandate, or government direction. 


1.39 Our review found that one of the sector business plans had no mission 
statement, while others differed in what is expected, from “a provider of 
information” to “a sustainable development organization.” A corporate 
strategic plan could help to provide a rationale to explain these differences. 


1.40 We noted that one of the five operational sectors had systematically 
reviewed its legislative authorities to identify gaps, overlaps, and areas where 
it was not meeting its legislative requirements or where it was doing 
unnecessary work. 


1.41 Government priorities can be derived from a variety of sources, such as 
the Prime Minister’s mandate letter to the Minister, the Speech from the 
Throne, and budget documents. We found no consistency in the documents 
used by the five operational sectors to determine government priorities. 
Sector priorities were based on 10 different government documents from 
three different years. Three of the five operational sectors had determined 
individual strategic priorities, while two sectors had aligned their business 
plans to specific issues. As a result, there are inconsistencies in the 17 
different strategic priorities formulated by the five operational sectors. For 
example, the Department’s 2004 Sustainable Development Strategy states 
that sustainable development is central to its mandate. Thus, we would have 
expected the business plans of the five operational sectors to identify 
sustainable development as a priority and demonstrate how each sector 
would contribute to it. We found that only two of the five operational sectors’ 
business plans referred to sustainable development. Public safety is the only 
common strategic priority identified across the Department. 


142. Climate change is another key government priority relevant to the 
Department. Natural Resources Canada has many program initiatives related 
to climate change and has informed us that it is heavily involved in all aspects 
of this work—developing policy, facilitating stakeholder consultations, 
supporting international negotiations, and creating significant new 
knowledge capacity. An overall corporate strategic plan could help to clarify 
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the importance of this initiative and other key government initiatives to the 
Department. 


143 Report on plans and priorities. We compared the strategic outcomes 
reported in the Department's report on plans and priorities with the five 
operational sector business plans. Two of the five operational sectors have 
aligned their business plans with these strategic outcomes, but the other three 
have not. Each sector determines how it will contribute to the attainment of 
these strategic outcomes. Sector business plans need to demonstrate 
alignment with the strategic outcomes in the report on plans and priorities to 
show how they will be achieved. 


144 Natural Resources Canada needs a consistent methodology and needs 
to establish common terms of reference for developing its strategic priorities 
for each sector. Doing so wilt help the sectors interpret the Department’s 
mandate and the government’s priorities. 


145 Recommendation. Natural Resources Canada should develop a 
corporate strategic plan to ensure that all sectors address key aspects of its 
legislation and government priorities. To do so, it should implement a 
systematic strategic planning process across all sectors. 


Department’s response. Natural Resources Canada agrees that a sound 
strategic planning process is an important tool to address government 
priorities. To that end, steps have been taken to strengthen the corporate 
planning activities of the Department, including placing a renewed focus on 
corporate planning within the newly restructured Strategic Policy Branch and 
the formulation of appropriate options for senior management consideration. 


Governance processes need a more strategic focus 


1.46 Good corporate governance is key in the development of corporate 
strategies. An effective governance mechanism ensures that strategies are 
reflected in organizational business plans and monitors the implementation of 
the strategies to assess their effectiveness. The Department has identified a 
need to improve its corporate governance. 


1.47 In 2001 it reviewed its governance practices to clarify roles and 
responsibilities and to identify options to improve efficiency and effectiveness. 
The review reported on the concerns of members of the Departmental 
Management Committee (DMC). It found that “the DMC was not a 
strategic decision-making governance body but rather an exchange forum 
on management issues.” Further, the review found that the managers of 
horizontal files generally needed to improve their understanding of roles and 
responsibilities. This report and our interviews with staff indicate that the 
DMC needs a more strategic focus. 


1.48 The Department’s 2003—04 Report on Plans and Priorities stated 

that it would review its departmental-level corporate governance system 

in 2003-04. As a first step, the DMC held three strategic retreats in 2003 
and 2004. It reviewed horizontal issues and decided that the heads of the 
Corporate Services Sector and Corporate Policy and Portfolio Coordination 
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Branch would undertake additional work to develop a more strategic agenda 
for the DMC. As we completed our audit, the Department had yet to develop 
this agenda. 


149 Executive performance agreements need improvement. Io ensure 
that departmental and sector objectives are implemented, key objectives are 
included in the individual performance agreements that each senior 
executive enters into annually with his or her supervisor. The individual 
performance objectives and the assessment of related results comprise an 
important part of the governance structure. We found that executive 
performance agreements aligned well with the sectors’ business plans. 
However, many of the performance agreements did not adequately address 
performance expectations. There was no process in place to discuss or share 
the performance agreements among the sector heads to ensure that the 
agreements adequately addressed horizontal issues involving more than one 


sector. 


1.50 For example, a key commitment in the 2003—04 Report on Plans and 
Priorities was to build a strong and diverse workforce. The Department 
indicated that this would require demographic analysis, forecasts of 
recruitment needs, an assessment of organizational health, and a need to 
monitor results. We found that the performance agreements did not contain 
any consistent description of the performance expectations related to this 
commitment. As a result, there was considerable variance in how the sectors 
had responded. One had plans for recruiting, learning, and career 
development in place, with appropriate milestones that are monitored 
regularly. Another had a draft plan, while a third had not yet assessed its 
staffing requirements. 


1.51 Sector business plans. As discussed, we found significant differences 
in priorities established in the individual sector business plans. Business plans 
and priorities of the sectors are not structured to bring the resources of the 
Department together to work on cross-cutting issues. In order to promote 
internal coherence and address cross-cutting issues, the Department needs to 
employ a common strategy; or, at a minimum, it needs to establish common 
terms of reference for developing the strategies in the sector business plans. 


1.52 Partially in response to this concern, Natural Resources Canada 
created the Office of the Chief Scientist. As we noted in paragraph 1.14, it is 
intended to bring consistency to the management of science and technology 
programs across the Department. Because the Office was just created in 
September 2003, it is too soon to assess its impact. There are other significant 
programs that cut across the Department that do not fall into the science and 
technology category. These programs are not reviewed and analyzed by an 
oversight body such as the Office of the Chief Scientist. 


153 Co-ordinating mechanisms. The roles, responsibilities, authorities, 
and accountabilities of sector heads seem well understood, both individually 
and collectively. The roles, responsibilities, co-ordinating mechanisms, 
authorities, and accountabilities for horizontal issues are less clear within the 
Department. 
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1.54 Managers view co-ordination and collaboration as an essential element 
for the successful management of horizontal issues. While informal bilateral 
agreements between sector and branch heads exist for some horizontal issues, 
other horizontal issues are managed by specific responsibility centres such as 
the Corporate Policy and Portfolio Coordination Branch, Office of the Chief 
Scientist, Corporate Services Sector, Communications Branch, a Shared 
Services Office, or an Office of Primary Interest (a manager or group of 
managers tasked with a particular responsibility). 


155 | Only rarely are horizontal responsibilities and relationships formalized 
through written agreements that specify the mandate and resources that 
accompany the assignments. Individual and shared accountabilities are not 
well understood or effectively communicated to all parties. For example, we 
found that one sector’s business plans indicated that it was spending about 
$61.5 million annually to support two other sectors. However, there was no 
communication between the sectors to ensure that the work planned was a 
priority for the other sectors. 


1.56 Recommendation. Natural Resources Canada should improve its 
governance processes. More specifically, it should 


* clarify the mandates, resources, roles, and responsibilities for the 
managers of horizontal issues; and 


* develop more strategic governance processes promptly to ensure that its 
strategies and actions are coherent and that they adequately address key 
aspects of its legislation and government priorities and are effectively 
monitored. 


Department’s response. Natural Resources Canada recognizes the need to 
improve its governance processes. The Department has undertaken an 
assessment of the Departmental Management Committee’s (DMC) 
governance processes, and options have been developed for the consideration 


of the DMC. 


Sectors lack appropriate industry information for strategic planning 


1.57 One of the Department's roles (identified in its legislation) is to gather, 
compile, analyze, co-ordinate, and disseminate information on matters 
affecting Canada’s natural resources. A key part of developing this knowledge 
requires information on the status of natural resource industries. Natural 
Resources Canada needs systematic, statistical, and economic forecasts for 
natural resource industries to enable it to assess sustainable development and 
develop its strategies. Not all sectors have adequate industry information. 


158 We found that the Minerals and Metals Sector has some good practices 
that could serve as a model for others. Annually, it publishes the Canadian 
Minerals Yearbook, which is both a review and outlook. The data gathering 
process is the same from year to year, thus providing comparative trends, and 
it involves the sector’s many commodity officers in conjunction with the 
Minerals and Mining Statistics Division. With delegated authority from 
Statistics Canada, the Minerals and Metals Sector collaborates with partners 


in planning, conducting, and co-ordinating a broad range of national surveys 
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Downstream industries—Those industries 
closest to the consumer. 


Upstream industries—Those industries 


closest to the resource. 


There is no precise cut-off between these 
two categories; some industries could be a mix. 


in the mining industry. The Minerals and Metals Sector has actively involved 
provincial ministries in the approval process to help ensure that the data are 
complete and accurate and to help reduce duplication of efforts. 


1.59 The Minerals and Metals Sector recently reviewed its legislative 
requirements for providing information and identified the need for the same 
degree of completeness and quality of data and analysis for the downstream 
industries (steel, automobile, and gas pipeline industries) as it currently has 
for the upstream industries (mining, smelting, and metal processing 
industries). 


160 The Energy Policy Sector last published an outlook document, Canada 
Emissions Outlook: An Update, in 1999. It also uses energy supply and 
demand forecasts produced by the National Energy Board. While the 
Canadian Forest Service has conducted statistical and historical analyses, it 
has yet to develop an outlook for its upstream and downstream industries. 


1.61 In 2003, Natural Resources Canada conducted a Department-wide 
study of the individual sectors’ overall capacities to scan the external 
environment and conduct statistical and trend analyses. The sectors had 
varying capabilities in this regard (Exhibit 1.6). 


Exhibit 1.6 Natural Resources Canada’s self-assessment of its analysis capacity 


Capacity for industry analysis 


Minerals 
Earth and 
Forestry Science Metals Energy 
Scanning the environmental, f 3 ; 
statistical, and trend analyses Meaiunt Pa Glen niep 


Source: Natural Resources Canada 


1.62 The Canadian Forest Service recently assessed what it is doing against 
what it believes it should be doing in the upstream and downstream forestry 
industries. It identified problems such as data gaps and inadequate human 
resources capacity in modelling, economic analysis, and policy development. 
For instance, the Canadian Forest Service lacks current data on domestic and 
international wood supply and international markets by product. While it has 
developed knowledge of specific issues, such as softwood lumber, it lacks 
in-depth analytical and technical ability. 


1.63 The Energy Policy Sector is aware that its previous statistical and 
economic analysis model covered demand for energy but did not address 
supply. At the time of the audit, the Energy Policy Sector had recently 
acquired a new model and was working on producing an outlook document. 


164 The Earth Sciences Sector is aware that it lacks the human resources 
capacity and data to conduct economic and industry analysis. 
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1.65 Natural Resources Canada needs to ensure that it has the information 
it needs to fulfill its legislative obligations and to inform its strategic planning. 
We found that some sectors had tried to obtain some information from 
sources outside the Department but were hampered by jurisdictional issues. 
Other sectors had not attempted to obtain information from outside the 
Department. 


1.66 Recommendation. Natural Resources Canada should determine and 
obtain the industry information that it needs to support strategic decision 
making by producing it internally or obtaining it from outside sources. 


Department’s response. Natural Resources Canada recognizes the 
importance of industry information to support strategic decision making and 
has long been active in seeking out, on an ongoing basis, industry information 
from a variety of sources, both internally and from outside sources. 


Human resources planning is inadequate 


1.67 Part of the strategic planning process involves the analysis of required 
human resources and developing a strategy to acquire them. A recent study 
by the Department found general dissatisfaction within the Department with 
the extent and reliability of human resources information available from the 
corporate human resource information system. We found that the 
Department does not have the information it needs to determine what staff it 
needs to meet its business requirements. 


1.68 Managers are aware that a substantial number of people will need to be 
replaced in the next five years. The demographic information (age, years of 
service, geographic distribution, and eligibility to retire by occupational group 
and level) gathered by the sectors from various sources varies significantly by 
sector in both the type of information and the level of detail available to 
develop a plan. One of the five operational sectors has an analysis and staffing 
plan that aligns staffing action with future business needs. Another sector has 
a draft staffing plan but has not assessed current competencies and aligned 
them with projected business needs. The three remaining operational sectors 
have only recently begun to analyze the skills and competencies required 
compared with those available. 


1.69 Recruiting science and technology staff. In 2004, the Department 

conducted a resources demand analysis of its science and technology staffing 
needs. Analysis results indicated that in certain key areas, the percentage of 
science and technology staff aged 50 or over was between 35 and 40 percent. 


1.70 More than 116 out of about 500 research scientists are eligible to retire 
immediately with a full pension. By April 2006, an additional 212 people in 
the Department will be eligible to retire; about half of those are scientists and 
technologists. While this study provided useful information about future 
retirements, it did not include information on current capacities and 


competencies or those needed to replace retiring staff. 
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1.71 Recruiting executive staff. Executive staff, primarily responsible for 
the leadership and direction of Natural Resources Canada, are also 
approaching retirement. A recent review of the executive category indicates 
that about 20 percent of about 120 staff will be eligible to retire in the next 
18 months. However, the competencies required to replace executive staff 
have not been determined, and there is no succession plan. 


1.72 Recruiting policy and economic analysts. Natural Resources Canada 
has a special recruitment program in place to strengthen its capacity to 
provide policy direction by hiring staff in the economic and social sciences 
service category. There are about 260 staff in this category at Natural 
Resources Canada. Between April 2001 and March 2003, 48 left. In 2004, 
the Department successfully recruited 22 qualified candidates with masters 
and doctoral degrees. The Department also has other recruitment programs 
aimed at meeting its needs in this area. 


1.73 Natural Resources Canada has recognized the importance of human 
resources planning for the Department as a whole. However, it needs to 
develop a plan and process to collect and maintain information on the 
education levels, skills, and competencies for staff in the research, policy, and 
management categories, particularly in the groups just below the senior levels 
that will be drawn on to replace retiring staff. This information needs to be 
integrated with labour market data and an analysis of the Department’s future 
needs in order to develop a recruitment and retention strategy that will 
ensure that Natural Resources Canada has the resources it needs to discharge 
its mandate. 


1.74 Recommendation. Natural Resources Canada should 
* document the competencies and capacities of its current workforce, 


* develop a clear understanding of the competencies and capacities that it 
will need to develop or acquire to replace the large percentage of key 
staff eligible to retire in the next few years, and 


* develop and implement a human resources plan that aligns the 
competencies required with future business needs. 


Department’s response. The Department agrees that it should develop and 
implement a human resources plan that aligns the competencies required 
with future business needs, recognizing the challenges implicit in the 
Department’s dependence on sunset funding. The “NRCan 2005-2008 
Strategies for the Management of Human Resources,” approved by the 
Departmental Management Committee, identifies specific recommendations 
for the Department to develop an integrated approach to the management of 
human resources. The Department’s strategies focus on the improvement of 
three significant areas: management/executive capacity, non-executive 
recruitment, and retention/well-being, as the Department prepares for the 
integration of human resources planning within the annual business planning 
cycle. 


18 


fl Chapter 1 


Report of the Auditor General of Canada—April 2005 


Preparing for emergencies and 
assessing risk 


NATURAL RESOURCES CANADA—GOVERNANCE AND STRATEGIC MANAGEMENT 


Information systems used for corporate oversight need improvement 


175 We reviewed the information systems used by the five operational 
sectors and five regional offices for decision making and corporate oversight, 
primarily in relation to scientific projects. Regional offices report through the 
operational sectors. We found that the Department’s information 
management practices are uneven and highly variable. 


176 Each sector and regional office we reviewed has created its own 
budgeting system and project management system for research and 
development. While some systems were integrated with the departmental 
financial information system, others had to be manually collated to produce 
information on the status of research and development projects. The systems 
were not integrated, and the Department was not able to provide us with a 
complete list of ongoing research and development projects. 


1.77 An important role of the Office of the Chief Scientist is to provide a 
framework for science and technology management. In order to achieve this, 
the Office needs consistent information across the sectors. The highly varied 
systems in use and the uneven data are currently inadequate to support the 
Office or to allow effective corporate oversight. Key information on research 
and development projects is needed across the sectors, including 


* the number of projects completed in the last five years; 
* the percentage completed on time, on budget, and on schedule; 
¢ the results compared against objectives; 


¢ the number of integrated projects and the processes by which they were 
integrated; and 


¢ the number of “post-mortems” and lessons-learned sessions conducted. 


1.78 Recommendation. Natural Resources Canada should improve its 
research project management systems to ensure that it has better-integrated 
information to support effective corporate oversight. 


Department’s response. The Department recognizes the need for 
improvements to its research project management systems. Following up on 
the Science & Technology (S&T) Information Needs and Options Analysis 
performed in 2003-04, a S&T Information System will be established to 
provide key information for all S&T activities within the Department at the 
sub-sub activities level of its Program Activity Architecture. The broad 
categories of information to be managed by this system can be categorized as 
operational information, performance information, resource information, and 


reach information. 


The Department is not prepared for emergencies in all its responsibility areas 


1.79 According to its March 2004 Performance Report, Natural Resources 
Canada spent $31.7 million to provide Canadians with safety and security in 
the natural resource sectors. Activities in this area relate to 

* meeting Canadians’ safety and security needs; 


* policy research and development and program support; 
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« regulation and assistance for the development of offshore oil industry; 
and 


* regulation, research, and control of explosives. 


1.80 Many of the Department’s ongoing activities contribute to the safety 
and security of Canadians. We focussed on its responsibility for emergency 
preparedness to deal with hazards that are identified in legislation and 
government direction. We reviewed this responsibility to determine how 
emergency preparedness is managed across the Department. 


1.81 The Emergency Preparedness Act establishes the requirement for 
Natural Resources Canada to have civil emergency plans for contingencies 
that are within or related to its area of accountability. A civil emergency plan 
developed pursuant to the Act is required, as appropriate, to provide for 


* assistance and advice to provincial governments and, through provincial 
governments, to local authorities; 


* federal-provincial regional plans; and 


- the safety and welfare, during an emergency, of officers and employees of 
the government institution. 


The Act also requires the Department to conduct training and exercises in 
relation to a civic emergency plan developed pursuant to the Act. 


1.82 In 1995, the government established a policy for emergency 
preparedness and named Natural Resources Canada as the federal lead in the 
following areas: 


* offshore oil and gas production and exploration in areas over which the 
Minister of Natural Resources has administrative authority (includes the 
Canada—Newfoundland and Canada—Nova Scotia oil and gas offshore 
accord area emergency plans); 


* mine disasters; 


* federal advice and assistance in combatting and containing fires, blights, 
disease, insect infestations, or other threats to Canada’s forest resources; 


* energy shortages or major power failures; and 


* control and regulation, in collaboration with the National Energy Board, 
of the production, generation, processing, transmission, storage, sale, 
domestic distribution, and exports and imports of energy. 


The Department informed us that this policy is out of date and is being 
revised. However, until it is replaced, it remains in effect. Government 
guidelines related to this policy are set out in Exhibit 1.7. 


183 In 1996, the government designated Natural Resources Canada 
together with Industry Canada as the lead for developing emergency plans for 
contingencies resulting from, or in, a shortage of strategic mineral 
commodities. 
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Exhibit 1.7 Government guidelines for emergency preparedness plans 


Emergency preparedness plans should provide for the following: 


Other jurisdictional co-operation—Secure to the extent possible and desirable the 
co-operation and active support of the private sector, benevolent and volunteer 
organizations, territorial and provincial governments and, through the latter, local 
authorities. 


Governance framework—Establish effective mechanisms of consultation, reporting, 
and overall direction and control, including other federal support departments. 


Reactivation priorities—Establish and administer priorities for the repair, replacement, 
rehabilitation, or reactivation of lost or damaged resources or facilities. 


Situational determination—Assess and determine regional and national requirements, 
in relation to available resources, for the supply of services and materiel. 


Resource allocation—Establish and administer priorities to ensure the effective 
allocation of services and materiel in short supply, and establish and maintain 
programs to overcome the shortages. 


Regional adaptation—Establish and manage administrative mechanisms or facilities 
necessary to permit the emergency response function to be effectively carried out in 
any region of Canada. 


International obligations—Co-ordinate with relevant international organizations to 
deploy resources to other countries in response to requests for emergency assistance or 
to fulfill international commitments. 


Worker safety and welfare—Ensure the safety and welfare, during an emergency, of 
federal employees and other responders working under departmental control. 


Source: Summary of Departmental Planning Responsibilities for Emergency Preparedness, June 1995 


184 The 1995 Federal Policy for Emergencies states, 


The federal government intervenes in civil emergencies only 
when formally requested by a province or territory or when the 
emergency clearly impacts on areas of federal jurisdiction, such 
as fires or floods on federal lands. .. . When the federal 
government does intervene, unless a lead minister has been 
designated in advance, a minister, and hence department, may 
be named to assume the lead role and co-ordinate the collective 
effort of the federal government. 


1.85 We reviewed Natural Resources Canada’s plans for emergency 
preparedness in its areas of responsibility. Our assessment of the essential 
elements for these plans is set out in Exhibit 1.8. We assessed the element as 
“fully met” when the plan explicitly addressed the element and included 
formal arrangements. Where a plan does not explicitly address the element 
and its arrangements, or where the references are vague, we assessed the 
element as “does not meet.” Two of the plans we reviewed were well-done 
and could be used as a model for developing plans in other areas of the 
Department’s responsibility. Other plans had some of the information 
required. The Department has no plans to address mine disasters or shortages 


of strategic mineral commodities. 
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186 In the area of forest resources, the Department helped establish the 
Canadian Inter-Agency Forest Fire Centre that co-ordinates forest-fire 
fighting ‘equipment across Canada. However, the Department was unable to 
provide us with plans to address the other requirements for forest resources. 


1.87. In August 2003, large portions of Ontario and the Midwest and 
Northeast United States experienced an electric power blackout. Natural 
Resources Canada was appointed as the Canadian lead on the Canada—U.S. 
task force that was established to investigate the causes of the blackout and to 
find ways to reduce the possibility of future outages. 


188 In spite of the 2003 blackout, at the time of our audit the Department 
was unable to provide us with plans to address energy shortages. 


1.89 Since the blackout, the Department has attempted to assess the risks to 
electricity supply but has found the task very difficult as it must depend on 
the voluntary co-operation of electricity suppliers. There are no current 
federal regulations that would compel the industry to co-operate. 


Risk assessment processes are inadequate 


190 An important part of emergency preparedness is assessing the possible 
risks in any of the Department’s responsibility areas. Risk assessment is also 

required throughout the organization for strategic planning and management. 
A risk assessment process allows the Department to answer these questions:: 


¢ What is the threat? 


* What is the probability of occurrence within a specified geographical 
area and a set time frame? 


¢ What are the economic, social, and environmental impacts? 


1.91. Risk assessments should be based on scientific and economic 
information and analysis. A standardized, systematic, and rigorous risk 
assessment system will provide the scientific basis to proactively manage 
threats, set priorities, develop plans, and allocate resources. In September 
2003, the Department released “A Guide for Using Natural Resources 
Canada’s Priority Setting Model to Rate Management Initiatives.” It provides 
guidance for risk assessment, but it is too general to provide effective 
guidance for assessing risks and threats related to potential civil and national 


emergencies. 


192 Each sector in the Department has dedicated resources to carry out 
scientific risk assessments. These assessments provide valuable information; 
however, sectors use different frames of reference in analyzing risks 
(Exhibit 1.9). This inconsistent approach makes it impossible for the 
Department to compare potential economic and social impacts across the 
sectors and best allocate departmental resources in either its day-to-day 


business or in the case of emergencies. 
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Exhibit 1.9 Examples of differing frames of reference 


Time frame used to analyze 


. probable frequency of 
Hazard Area analyzed to assess impact occurrence 
Wildfires (Canadian Four square kilometres 41 years 


Forest Service) 


Explosives (Minerals and Administrative regions 2 years 
Metals Sector) 


Natural Hazards (Earth All of Canada Varies by hazard 
Sciences Sector) 

Energy Infrastructure Company service area (for Frequency not 
(Energy Policy Sector) companies who have considered 


volunteered information only) 


1.93 It is particularly important that the Department have information on 
emerging threats. While the sectors continue to develop valuable contacts 
and to identify new sources of information, this intelligence comes from a 
number of sources and is gathered informally. The information is not 
integrated into a formal risk assessment process. 


Economic information and analysis needs to be strengthened 


1.94 A good risk assessment should include analysis of the possible 
economic impacts that may result when potential threats actually occur. The 
quality and nature of the economic information that is used has a direct 
impact on the value of the completed risk assessment. We looked at examples 
of certain threats and found that the assessment of economic impacts of the 
threat could be strengthened. We found that when assessing economic 
impacts, the sectors considered a range of factors such as departmental 
liability, loss of reputation, overall personal injury and property damage, and 
future loss to society. However, the sectors did not consistently consider the 
same factors, and they have attempted to quantify the consequences in 
different ways. 


1.95 More specifically, there was limited information on who would be at 
risk, the volume and value of the property at risk, and other values that might 
potentially be at risk, such as departmental liability or the general economy. 
Sources of economic information are not well developed, making it difficult 
to assess the economic impact of a threat. 


196 Recommendation. Natural Resources Canada should develop plans 
and improve its risk analyses to deal with its responsibilities in emergency 
situations. In particular, the Department should 


* develop, in co-ordination with other key stakeholders, emergency plans 
} gency | 
in the areas where it is required to respond to civil emergencies, 
particularly where it has the lead federal role; and 
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* re-examine its risk assessment guidance and apply risk assessment 
processes consistently across its five operational sectors, including 
\ + + . . o . 
applying a more coherent analysis of the economic impact of potential 
threats. 


Department’s response. Natural Resources Canada (NRCan) recognizes the 
importance of appropriate civil emergency plans. The document that 
identifies the responsibilities for federal departments, “Departmental 
Planning Responsibilities for Emergency Preparedness,” has not been 
amended since its 1995 publication, despite significant changes to Canada’s 
national security environment in the last decade. Natural Resources Canada 
has engaged Public Safety and Emergency Preparedness Canada (PSEPC)— 
the successor department to that which authored the 1995 document. PSEPC 
agrees that the 1995 planning document is outdated and has advised that it 
has launched a process to modernize Canada’s emergency preparedness 
policy. NRCan will continue to work collaboratively with PSEPC in the 
development of appropriate emergency plans in the context of evolving 
federal policy. 


The Department is strengthening its integrated risk management program to 
better integrate risk management into the Department's activities and, in so 
doing, it will develop further guidance for the sectors on the application of 
risk assessment methodologies and tools within NRCan. 


The Department wishes to respond to the statements made in paragraphs 
1.87, 1.88, and 1.89, to clarify the role and nature of NRCan’s involvement . 
in electricity reliability. The governments of Canada and the U.S. agreed 

in 1969 that the North American Electric Reliability Council (NERC) was to 
develop reliability standards for the electric system in Canada and the U.S., to 
monitor compliance with the standards, and to establish procedures for 
emergency support for electric utilities experiencing unanticipated reliability 
difficulties. This system was in place in August 2003, and it succeeded in 
restoring power to much of the blacked out region within hours and to the 
entire region within days. NRCan has since worked with the U.S. in a task 
force to analyze and report on the blackout. The task force made 

46 recommendations to improve electricity reliability. NRCan, NERC, 
industry, regulators, and provincial governments, who have jurisdiction over 
electricity, are working together to implement these recommendations in 
Canada. Further, pursuant to Canada’s commitments under the Smart Border 
Declaration, signed in December 2001, NRCan initiated a process with the 
U.S. to conduct vulnerability assessments of critical cross-border energy 
infrastructure. The pilot assessments of electricity generation and cross- 
border transmission infrastructure were completed in March 2004. 
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Information for Parliament 


1.97 Natural Resources Canada is accountable for tabling four major annual 
reports in Parliament: 


* Report on Plans and Priorities; 


¢ Departmental Performance Report; 


Improving Energy Performance in Canada; and 


« The State of Canada’s Forests. 


It is also required to table a report on its sustainable development strategy 
once every three years. We did not review the Department’s performance 
against its sustainable development strategy. 


198 We reviewed the 2001—02 and 2002-03 reports titled Improving 
Energy Performance in Canada; the 2002—03 and 2003-04 reports titled 
The State of Canada’s Forests; the March 2003 and March 2004 
departmental performance reports; and the reports on plans and priorities 
for 2003-04 and 2004—05. We also reviewed supporting documentation for 
the reports and interviewed responsible officials. We did not audit the data 
in the reports. 


The Department generally meets its annual reporting obligations 


1.99 Reports on the administration and enforcement of the Energy 
Efficiency Act. As soon as possible after the end of each fiscal year, Natural 
Resources Canada is required to report to Parliament on the administration 
and enforcement of the Energy Efficiency Act. The Act deals with the 
importation and inter-provincial trade in energy-using products (for example, 
refrigerators) and the promotion of energy efficiency and alternative energy 
sources. It also provides authority for the government to make regulations 
with respect to matters related to energy and energy-using products. 


1.100 While the reports include substantial and detailed information and 
provide examples of good reporting practices, they could be improved in their 
timeliness and coverage (for example, regulatory coverage, enforcement 
provisions, and energy efficiency of imports). 


1.101 Reports on the state of Canada’s forests. Natural Resources Canada 
is required to report annually to Parliament on 
* the state of the forests in Canada, 


* the contribution of Canada’s forests to the economy and environment 
of Canada, and 


* the contribution of Canada’s forests to the social-well-being of Canadians. 


1.102 We found that the Department has generally met its reporting 
obligations and has improved the report and made it more useful to readers. 


Performance reporting has several shortcomings 


1.103 In the case of the departmental planning and performance reports, we 
expected the Department to produce adequate performance information, use 
this information to manage for results, and demonstrate good reporting of 
these results to Parliament. 
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1.104 No coherent process for performance management. Like many other 
departments, Natural Resources Canada has found it difficult to establish 
results at a corporate level and to manage its sectors to achieve those results. 
We found no systematic process that could help the Department move 
toward this at a Department-wide level. The Department’s performance 
measurement framework, approved by the Treasury Board in 1999, comprises 
5 departmental goals (or strategic outcomes), 16 short or medium-term 
objectives, and 35 performance indicators. 


1105 At the time of our audit, the Department was revising its performance 
measurement framework. It is too early to say whether the proposed changes 
will result in a better framework. 


1.106 The Department’s reports on plans and priorities and its 

performance reports are largely a sum of the sector plans and reports. 

In the two operational sectors that have aligned their business plans with the 
departmental plans, assessments of results achieved against the business plans 
have helped provide good information for the departmental reports. In the 
three other operating sectors, the lack of alignment with the departmental 
reports generally precludes good reporting. 


1.107. Reports need improvement. The Department’s performance reports 
set out a brief vision statement and five strategic outcomes. Because the 
reports do not include a statement of the Department's legislated mandate or 
of its mission, it would be difficult for parliamentarians to determine whether. 
the Department’s planned strategic outcomes are consistent with its 
mandate. 


1.108 Although the reports link the Department’s priorities to government 
priorities, they do not include a specific discussion of the contribution of the 
strategic outcomes to the government's priorities. Moreover, the reports do 
not include any discussion of risks or challenges to achieving the strategic 
outcomes identified by the Department. 


1.109 In the reports on plans and priorities, each strategic outcome aligns 
with broad statements of commitments and key accomplishments. In the 
March 2003 Performance Report, the key accomplishments are described 
mostly in narrative form in terms of activities, outputs and, in some cases, 
outcomes. The March 2004 Performance Report also includes a chart that 
summarizes performance against each commitment made in the related 
Report on Plans and Priorities. 


1.110 The Department made a commitment in its 2001-02 Report on 

Plans and Priorities to report on 35 performance indicators on a cyclical basis. 
That is, each performance report since 2001-02 has included data on a 
different grouping (about one quarter) of the 35 performance indicators. 
The March 2003 Performance Report did not explain the cyclical reporting. 
This lack of complete performance data and assessments against various 
frameworks makes it difficult for parliamentarians to get a complete picture 


of the Department’s performance. 
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1.111 Recommendation. Natural Resources Canada should improve its 
performance measurement framework and reporting to provide 
parliamentarians with better information on the results and outcomes of its 
programs. In particular, it should 


* develop a systematic process for linking performance targets at the 
corporate level with business plans; and 

* improve its performance reports by including an analysis of its legislated 
mandate, linking the Department's priorities to government priorities 
and rationalizing its assessment of its performance against various 
frameworks. 


Department’s response. Natural Resources Canada agrees that it needs to 
better explain its performance measurement reporting at the corporate level. 
As part of the new Program Activity Architecture, the Department will 
provide a better linkage between corporate performance targets and the 
business plans of its program activities. 


Natural Resources Canada will include mandate and mission information in 
all future reports on plans and priorities and departmental performance 
reports, and it will better explain the linkages between the government’s and 
the Department’s priorities. 


Conclusion 


1.112 In recent years, Natural Resources Canada has focussed on a number 
of major initiatives. However, we found that its strategic governance 
mechanisms, management systems, and practices need improvement. 


1.113 The Department does not have a corporate strategic plan or a coherent 
Department-wide process to consider its legislated mandate and potential 
risks in the development of its strategies. Departmental strategies are not 
adequately reflected in the sectors’ business plans and communicated 
throughout the Department. 


1.114 In the one strategic priority that was consistently identified as a priority 
across the five operational sectors—public safety, we found little consistency 
in the sectors’ analysis of their responsibilities for emergency preparedness or 
in the adequacy of how they dealt with it. 


1.115 The Department does not have adequate systems for aligning qualified 
staff with current and future strategic needs, nor does it have a plan to ensure 
that it will maintain the human resources capacity it needs to deliver on its 
mandate and programs in the short term. We also found weaknesses in the 
information management systems for supporting oversight at the corporate 
level. 


1.116 Natural Resources Canada generally meets its statutory obligations 
for key annual reports. However, we found that the Department needs to 
improve its processes for assessing performance information on its key 
activities and the results of its efforts and for reporting this information 
to Parliament. 
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About the Audit 


Objectives 
Our audit objectives were 


* to assess whether Natural Resources Canada’s governance mechanisms, management systems, and practices at 


the strategic level are adequate to support its Department-wide policy development, research, program, and 
other activities; and 


* to determine whether the Department appropriately assesses and reports to Parliament the results and impacts 
of its efforts and meets its statutory reporting obligations. 


Our more specific objectives were to determine whether Natural Resources Canada 


* periodically reassesses the appropriateness of its activities; makes its strategic decisions with adequate 
consideration of legislated mandate, risks, options, stakeholders, and potential impacts, and communicates 
them effectively; and aligns its business plans with the departmental strategy; 


+ has appropriate systems and practices for selecting and terminating programs and research and development 
projects, and ensuring adequate oversight at the corporate level; 


* aligns qualified staff with current and future strategic needs; 


* uses performance information on its key activities to effectively monitor and manage its operations and report 
to Parliament on the achievement of its objectives; and 


* meets its reporting obligations pursuant to the Report on the State of Canada’s Forests Regulations and the Energy 


Efficiency Act. 


Scope and approach 


The audit focussed on Natural Resources Canada’s current governance and management practices, systems, and 
procedures that are used to manage its policy analysis, regulatory, program, and science and technology business 
activities at the strategic level. Our audit covered the Department’s industry-based sectors and its Earth Sciences 
Sector; the latter provides support to the industry-based sectors through its geoscience and geomatics activities. 


The audit also included examining the information management and human resource planning activities of the 
Department’s corporate branches: Corporate Policy and Portfolio Coordination Branch, the Office of the Chief 
Scientist, and the Corporate Services Sector. 


Our audit process included interviews with about 140 management staff and key stakeholders. It also included 
extensive review and analysis of the following documents, systems, and practices: 


* authorities, strategies, business plans, policies, estimates, directives; 
* minutes and observations of internal management meetings and external advisory meetings; 
* major management and control systems and procedures; 


- departmental performance reports and reports on plans and priorities as well as newer, additional mechanisms 
for reporting, such as Internet links to auxiliary departmental information; and 


* management accountability agreement commitments and evaluations. 


We visited the following regional offices: Northern Forestry Centre, Edmonton, Alberta; Geomatics Canada, 
Edmonton, Alberta; Devon Pilot Plant, Devon, Alberta; Pacific Forestry Centre, Victoria, British Columbia; and 


Pacific Geoscience Centre, Victoria, British Columbia. 
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Criteria 


The Department should have a strategic plan consistent with its mandate and government priorities that gives due 
consideration to factors such as external environment, risks, options, stakeholders, available resources, 
organizational strengths and weaknesses, and potential impacts. The strategic plan should align risks with strategic 
options, proposed courses of action, objectives, and expected results. It should be effectively communicated to staff 
and implemented throughout the organization. 


The Department should have long-term plans and business plans with clear objectives, goals, and expected results 
consistent with its strategic plan. These plans should be used to manage for results. 


The Department should have a risk management framework, systems, and practices that ensure that 
* appropriate information is available for decision making on strategic priorities and resource allocations; 
* business risks are managed; 
* programs, initiatives, and projects are integrated; 
* good project management practices are used; and 


* corporate oversight of progress and results is adequate. 


The Department should have appropriate human resource planning processes and strategies over a forecasted 
five-year period that 


* analyze its current workforce (especially research, policy, scientific and management staff and their feeder 
groups) in terms of demographics, skills, and competencies; 


* assess the future renewal requirements of the above-noted groups in light of their strategic direction; 


* identify gaps in skills needed and currently available, now and in the future, and identify over-supply, if any, in 
current workforce; 


* specify strategies and accountabilities for addressing the gaps or over-supply and include an action plan that 
integrates recruitment, retention, learning, and career development; and 


* provide for reports to senior management on progress and updates of the human resource plan on a regular 
basis. 
Natural Resources Canada should have a performance management framework and management practices that 
* produce adequate performance information, 
* use this performance information to manage for better results, and 


* demonstrate good public reporting of these results to Parliament. 


Natural Resources Canada should table a report on the state of the forests in Canada in each House of Parliament 
annually. The report should include information on the contribution of Canada’s forests to the economy and 
environment of Canada, and the contribution of Canada’s forests to the social well-being of Canada. 


Natural Resources Canada should table a report as required by the Energy Efficiency Act, in each House of 
Parliament annually. The report should include information on 


+ the administration and enforcement of the provisions of the Energy Efficiency Act, during the preceding fiscal 
year and regulations made under that Act related to energy-using products; and 


* the administration and enforcement of the provisions of the Energy Efficiency Act, during the preceding fiscal 


year and regulations made under that Act related to promotion of energy efficiency and alternative energy 
sources. 
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Audit team 


Assistant Auditor General: Nancy Cheng 
Principal: Crystal Pace 


Lead Director: Angus Mickle 


Robert Taylor 
Carol McCalla 
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For information, please contact Communications at (613) 995-3708 or 1-888-761-5953 (toll-free). 
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we also draw upon the standards and practices of other disciplines. 
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National Security in Canada— 
The 2001 Anti-Terrorism Initiative 


Air ‘Transportation Security, Marine 
Security, and Emergency Preparedness 


Main Points 


21 In December 2003—two years after it launched its anti-terrorism 
initiative—the government reorganized federal emergency planning, creating 
Public Safety and Emergency Preparedness Canada. The new department 
began an extensive reform program, but much remains to be done. Threat 
and risk analyses were not always used to guide the disbursement of funds by 
the previous agency to municipalities and provinces, resulting in poor 
allocation of funds. We found in a number of cases that programs do not 
appear to have achieved the intended results: 


* In 2002 to 2004, federal funds were used to purchase equipment for first 
responders that did not have to be interoperable, losing a key 
opportunity to enhance the national capacity to respond to an 
emergency. 


* The Office of Critical Infrastructure Protection and Emergency 
Preparedness, responsible for implementing a critical infrastructure 
protection initiative of Budget 2001, could not tell us what it had spent 
funds on. About one third of the funds allocated so far to this program 
may have lapsed. 


2.2 Governance structures require continued attention. The government 
has established the Government Operations Centre to manage emergency 
responses, but the legislative and infrastructure frameworks remain 
uncompleted. Managing the budgets of projects that cut across several 
departments worked well in some cases, such as marine security and the 
CBRN Research and Technology Initiative, but failed in the cases of CBRN 
training for first responders and critical infrastructure protection. (CBRN 
refers to chemical, biological, radiological, or nuclear threats.) 


23. The departments responsible for the CBRN Training Initiative 
estimated that about 6,000 first responders should be trained in how to 
intervene in and neutralize a serious event. We found that 200 people had 
been trained to this level so far under the CBRN Training Initiative. 


24 Some key elements of the federal government's air transport security 
program are moving forward. The purchase and installation of explosives 
an initiative by the 


detection systems to screen passengers and baggage 
Canadian Air Transport Security Authority (CATSA) at a cost of over 
$1 billion—are proceeding mostly as planned. Analyses of requirements and 


options were generally adequate. 
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25 Other elements, however, are working less well. Transport Canada told 
us it does not have a major problem with CATSA, but the Department has 
set no system-wide performance levels for CATSA and completed no 
assessments of its performance. The Department has not defined the 
sanctions it will use against CATSA if education and persuasion fail to get 
timely results. 


26 Marine security programs have been based on an adequate threat and 
risk analysis. Implementing the Automated Identification System for ships 
encountered delays but is still expected to be completed by its original 
implementation date. The High Frequency Surface Wave Radar project, 
intended to provide continuous, real-time surveillance of Canada’s coastal 
areas, represents an improvement in surveillance but will not provide full 
coverage. The government is acquiring fewer radar sets than considered 
necessary for complete coverage, and the radar will have a more limited range 
than originally forecast. The result will be gaps in coverage, with significant 
additional costs if full coverage is to be attained. 


27. The government’s implementation of the International Ship and Port 
Facility Security Code, necessary to keep Canadian ports and ships in the 
business of world trade, is generally going well. However, some certificates 
were issued on an interim basis, and security inspectors may be pressed to 
complete the inspections required for new certificates by the renewal 


deadline. 


Background and other observations 


28 This is the second of two audit reports on the federal government’s 
anti-terrorism initiative announced in the 2001 Budget after the 
September 11 attacks on the United States. The government initially 
announced $7.7 billion for this initiative and subsequently added another 


$690 million. 
29 Our first report, published in March 2004, addressed budget 


management, the allocation of funds, monitoring of expenditures, 
intelligence issues, and some aspects of air and border security. This chapter 
addresses air transportation security, elements of marine security, and 
emergency preparedness programs. 


The departments and agencies have responded. In general, the 
organizations have agreed with our recommendations, although 
commitments toward remedial action remain vague in some cases. We found 
the response from Public Safety and Emergency Preparedness Canada to be 
positive, as the Department is already moving ahead in some areas. 
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Introduction 


210 This chapter is the second of two on the government's Budget 2001 
national security enhancement initiative. The first chapter was published in 
the March 2004 Report of the Auditor General of Canada (Chapter 3, 
National Security in Canada—The 2001 Anti-Terrorism Initiative). Excerpts 
from that chapter are provided as background in Exhibit 2.1 


211 Previous audit findings. Our 2004 chapter addressed budget 
management, the allocation of funds, monitoring of expenditures, intelligence 
issues, and some aspects of air and border security. We reported the following: 


* The government lacked a management framework to guide investment, 
management, and development decisions and allow it to direct 
complementary actions of separate agencies or choose among conflicting 
priorities. 


¢ The government had failed to improve its security information systems 
to ensure that they could communicate with each other. 


¢ We found a lack of co-ordination of intelligence and a failure to 
adequately assess the intelligence lessons learned from critical incidents 
such as the September 11 attacks. 


¢ Lists used to screen visa applicants, refugee claimants, and travellers 
entering Canada had gaps and inconsistencies. 


* Criminal intelligence data were not used in screening applicants for 
clearance to restricted areas at airports. 


The government responded with actions to address our findings. We have not 
yet followed up on its response. 


212 The Government of Canada released its first national security policy 
on 27 April 2004. The policy sets out an integrated approach to security 
issues across government to address a wide range of risks and threats. The 
government allocated an additional $690 million from the contingency 
reserves of Budget 2001 and Budget 2003 and provided new funding in 
Budget 2004. The policy focussed on three national security interests: 


* protecting Canada and Canadians at home and abroad 
* ensuring that Canada is not a base for threats to our allies 
* contributing to international security 


213. The policy sets out actions planned by the government in a number of 
key areas: 

- intelligence 

* emergency planning and management 

* public health 

* transport security 

* border security 


* international security 
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Exhibit 2.1 Federal response to September 11, 2001 


Excerpts from March 2004 Report of the Auditor General, 
Chapter 3—National Security in Canada—The 2001 
Anti-Terrorism Initiative 


3.8 On September 11, 2001 the United States suffered an 
unprecedented terrorist attack that destroyed the World 
Trade Center, damaged the Pentagon, destroyed four 
civilian airliners, and killed thousands of citizens. The 
immediate effects on Canada were the need to deal 
with the shutdown of civil air transport and look after 
passengers on grounded airliners; heightened border 
security; and a sudden sense of personal and economic 
insecurity. 


3.9 The crisis period lasted several months, during which 
the federal government had to sustain internal and 
border security operations at a high level. Defence, 
intelligence, police, and border control agencies worked 
to full capacity. Ministers and senior managers sought 
to deal with policy and budget issues on an urgent 
basis, while at the same time drafting emergency 
legislation and guiding it through Parliament. 


3.10 In the longer term, the federal government has had to 
develop policies and programs to deal with the threat of 
terrorism not only to Canada directly but also to the 
United States and the rest of the world. 


3.11 Management of national security. On 12 December 
2003, the Prime Minister announced significant 
changes to the structure of parliamentary committees, 
departments, and agencies. The principal changes 
involving national security were the following: 


¢ Anew department, Public Safety and Emergency 
Preparedness Canada, was created from the former 
Solicitor General Canada. The new department 
includes the Office of Critical Infrastructure Protection 
and Emergency Preparedness, transferred from 
National Defence. 


The Canada Border Services Agency, reporting to the 
Minister of Public Safety and Emergency 
Preparedness, comprises the Customs Branch from 
the former Canada Customs and Revenue Agency, the 
intelligence and enforcement sections from 
Citizenship and Immigration Canada, and the border 
inspection function of food, plant, and animal health 
from the Canadian Food Inspection Agency. 


The new position of National Security Advisor to the 
Prime Minister in the Privy Council Office co- 
ordinates integrated threat assessments, helps 
strengthen interagency co-operation, and assists in 
the development of an integrated policy framework 
for national security and emergencies. 


ah dtG) 


3.20 


¢ The Minister of Transport is now responsible for 
security in all transportation sectors. 


* A new Cabinet Committee on Security, Public Health 
and Emergencies manages national security and 
intelligence issues and activities and government- 
wide responses to public health, national disasters, 
and security emergencies. It replaces the Ad Hoc 
Committee on Public Security and Anti-Terrorism. 


* * * * * 


Until December 2003, no single minister below the 
Prime Minister was responsible for Canada’s security. 
The organizations involved in security reported to their 
respective ministers, who were accountable for their 
activities. Ultimately the Prime Minister was, and 
remains, accountable for the security of the country and 
therefore provides broad guidance. 


* * * * * 


New funding. During October 2001, the government 
approved several major new allocations of funds, 
including: 


¢ $30 million annually to provide immediate, 
permanent staff increases to the Canada Customs 
and Revenue Agency, Citizenship and Immigration 
Canada, the RCMP and Transport Canada; 


$250 million for immediate security initiatives— 
largely capital and equipment—to 15 departments 
and agencies; 


$71.5 million in urgent funding to offset unforeseen 
costs such as overtime for Customs and the RCMP; 
and 


$160 million to compensate Canadian air carriers 
and specialty operators for losses resulting from the 
closure of Canadian air space following the 
September 11 attacks. 


Except for the funds to compensate air carriers, these 
amounts were part of the $7.7 billion announced in the 
December 2001 Budget as new spending over 
2001-02 and the following five years for enhanced 
security, emergency preparedness, and improving 
border infrastructure. The Budget was designed to keep 
Canada safe, keep terrorists out, and keep Canada’s 
border open. It announced $6.5 billion for security, 
including the creation of a new air security authority, 
additional funding for intelligence and policing, and 
funding for Canada’s military; and more than 

$1.2 billion for initiatives designed to make Canada’s 
border more secure, open, and efficient. 
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Exhibit 2.1 Federal response to September 11, 2001 (continued) 


3.21 The Budget included major investments to 


* equip and deploy more intelligence and front-line 
investigative personnel, improve co-ordination among 
agencies, and boost marine security ($1.6 billion); 


improve screening of immigrants, refugee claimants, 
and visitors (including detention and removal), speed 
up the determination of refugee claims, and introduce 
new fraud-resistant Permanent Resident Cards 


($1 billion); 


* create a new air security organization, place armed 
plainclothes police officers on Canadian aircraft, 
purchase explosive-detection equipment, and 
enhance air transportation policing ($2.2 billion); 
and 


° enhance border security and improve the 
infrastructure that supports major border crossings to 
ensure the legitimate flow of goods and people 
($1.2 billion). 


improve the protection of critical infrastructure and 
emergency preparedness and response; and expand 
the military’s anti-terrorism capacity ($1.6 billion); 


Focus of the audit 


2.14 


Given the large number of programs and initiatives funded by the 


December 2001 Budget, we decided to examine air travel security, elements 


of marine security, and emergency preparedness in a second audit. This 


chapter reports the results of that audit. 


2.15 


The chapter focusses on three specific program areas funded from 


the 2001 Budget: 


2.16 


The integrity of the air transportation security system. Transport 
Canada sets policy for and regulates the air transportation security 
system. We looked at how well it plays its role. We also looked at 
whether the implementation of explosives detection systems at Canada’s 
airports was well planned and is proceeding smoothly. The 2001 Budget 
allocated more than $1 billion over five years for this purpose. 


Marine security. The 2001 Budget initiative identified marine security as 
a priority, but the government was unable to put forward a plan quickly. 
It formed the Interdepartmental Marine Security Working Group in 2001, 
which developed a multi-agency program. We audited two capital projects 
that were part of this initiative—High Frequency Surface Wave Radar 
and the Automatic Identification System for ships. We also audited the 
federal government's part in implementing the International Ship and 
Port Security Code, intended to secure Canadian ships and ports. 


Emergency preparedness. Budget 2001 allocated $513 million over 
five years to improve federal and national preparedness for chemical, 
biological, radiological, and nuclear attacks. We audited the 
accountability and management structure of the programs, the adequacy 
of the capacity being created, and the progress of training and exercise 


programs. 


More details on the objectives, scope, approach, and criteria of the 


audit are included at the end of the chapter in About the Audit. 
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Air transport security 


Observations and Recommendations 


Security inspection and enforcement roles and responsibilities 


217 Aviation security aims to protect the general public, passengers, crew 
members, airports, other aviation facilities, and aircraft against unlawful 
interference. Transport Canada has established a range of tools to promote 
security, including programs for prevention, detection, awareness, education, 
and deterrence. Security programs cover regulations that govern airport 
operators; air carriers; agencies that screen passengers and baggage before 
allowing them into the system; access to restricted areas; possession of 
weapons; persons under escort; and response to security threats. Transport 
Canada is responsible for enforcing its regulations. Its security inspectors can 
assess monetary penalties, remove a security screener’s authority, and detain 
aircraft on the ground or recall an aircraft in flight should there be an 
immediate threat to security. 


218 A second federal agency—the Canadian Air Transport Security 
Authority (CATSA)—plays a key role in security. Included in its 
responsibilities are the pre-boarding screening of passengers and the baggage 
and possessions they carry as well as the screening of airport workers entering 
restricted areas. CATSA can do this screening either directly or through a 
screening contractor. It is responsible for establishing the qualifications, 
training, and performance standards that security screeners must meet and 
for certifying that they meet them. 


219 Many other organizations play a part in security. The RCMP and local 
police are responsible for law enforcement; local airport authorities are 
responsible for the physical security of airports; air carriers implement 
regulations intended to secure passengers, baggage, freight, and aircraft; cargo 
companies are responsible for the security of their operations. 


2.20 ‘Transport Canada is the policy-maker and regulator and, as such, 
occupies the lead role in air transport security. 


2.21. The 2001 Budget allocated $2.2 billion over five years to strengthening 
aviation security. The most important measures taken by the government 
since September 11, 2001 include 


* creating CATSA, through legislation, to take over passenger screening 
from the air carriers and also to screen non-passengers; 


* deploying systems to detect explosives; 

* providing airports with grants to improve airport security and policing; 
* placing armed RCMP officers aboard some flights; 

* improving aircraft cabin security; 

* evaluating advanced technologies for airport security; and 


* enhancing regulatory and oversight capacity. 
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Creating and operating CATSA accounted for $1.9 billion of the $2.2 billion 
allocated for aviation security. The remaining $300 million was to fund all 
other air security improvements. 


Risk management has not been formalized 


2.22 ‘The Treasury Board Secretariat’s Integrated Risk Management 
Framework calls risk management “a systematic approach to setting the best 
course of action under uncertainty by identifying, assessing, understanding, 
acting on and communicating risk issues.” 


2.23 The Framework describes a process to identify issues, assess key risks, 
measure the likelihood and impact of risks and rank them accordingly, define 
the desired results, and select and implement strategies for managing or 
mitigating the risks (Exhibit 2.2). 


2.24 Our objective in auditing aviation security was to determine whether 
Transport Canada’s oversight of the air transport security system is adequate. 
We expected to find that the system was based on appropriate risk analysis 
and that Transport Canada was taking the necessary measures to ensure 
compliance with its security regulations. 


2.25 We found that Transport Canada’s approach to assessing security risks 
in air transport (to the extent that it has done so) is consistent with the 


Exhibit 2.2 A common risk-management process 


Identifying 
Issues, 
Setting 
Context 


Assessing 
Key Risk 


Monitoring, 


Evaluating 


and Adjusting 


Measuring 
Likelihood 
and Impact 


Practise 
Integrated Risk 
Management 


Implementing 
the Strategy 


From corporate strategy 
and plans to front-line 


Ranking 
Risks 


4 


operations... people and 
processes 


Selecting 
a Strategy 


Setting 
Desired 


5 Results 


Developing 
Options 


Source: Treasury Board Secretariat 
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Treasury Board Secretariat’s Framework. However, it has not fully 
implemented formal risk management. At the time of our audit, one region of 
the Department was piloting a risk-based approach to managing the 
inspection process. Transport does have a general threat assessment that is 
updated annually by its intelligence directorate, which also sends specific 
notices and alerts to airports as needed. However, we did not find any 
comprehensive assessment of key risks or any measuring of the likelihood or 
potential impact of specific threats. 


2.26 ‘Transport Canada has done some work that partly meets the 
requirements of a sound risk management approach: 


« An interdepartmental working group on aviation security was 
established in September 2003 to assess threats and determine 
appropriate action. However, its terms of reference limit it to assessing 
air carriers’ applications for new overseas air routes. The working group 
has not been asked to develop an overall assessment of threats to 
aviation security. 


¢ With the U.S. Transportation Security Administration, Transport 
Canada has assessed the potential impact of a specific threat as well as 
vulnerability to that threat, overall and at particular airports. This 
assessment met Treasury Board criteria but was limited to a single threat. 


* Transport Canada relied on a series of teleconferences among senior 
security managers to deploy the additional security inspectors funded by 
the 2001 Budget and to supplement the existing minimum schedule of 
inspections. According to officials, they considered factors such as the 
size of airports, the number of flights, the riskiness of flight destinations, 
and the frequency of flights to the United States. While these decisions 
took into account the Department’s experience and understanding of 
intelligence information, they were not documented as a formal risk 
analysis. 


We also noted that the RCMP uses formal risk assessment in deploying its 
Aircraft Protective Officers aboard flights, except where carrying an officer is 
mandatory. 


2.27. ‘The assignment of inspection resources appears to be weighted toward 
risks from passengers and baggage. Risks from air cargo and general aviation 
(that is, small private and chartered aircraft) attract less scrutiny, despite the 
level of concern they have generated. Transport officials pointed out that they 
had doubled the number of cargo inspections and had implemented new 
programs of cargo security awareness with industry. Nevertheless, it is not 
clear to us that inspection resources and assignment of tasks have been 
allocated appropriately. Nor is it clear how funding for security inspection was 
allocated between major airports and smaller ones, when both are gateways to 
the entire system. Because the inspection program has not been based on a 
documented risk analysis, we could not conclude whether the number of 
inspectors and the frequency of inspections are appropriate. 
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2.28 ‘Transport Canada officials told us that they had not done a wider 
analysis before allocating the 2001 Budget funding for inspection, because the 
federal government had emphasized passengers as the key risk and the Budget 
had directed funding only to passenger transportation. The officials told us 
they intend to begin a complete review of security for all modes of 
transportation in 2005 in order to develop a security strategy. 


2.28 Recommendation. Transport Canada should complete a formal 
analysis of threats and risks to the entire air transport system and use the 
results as a basis for deploying resources and focussing enforcement efforts. 


Transport Canada’s response. Transport Canada recognizes the importance 
of risk management, which has been an essential foundation of its aviation 
security program since its inception in the 1970s. More recently, the 
Department has initiated a comprehensive Transportation Security Strategy, 
which will examine risk in all modes and activities within each mode. The 
strategy will include a formal threat-and-risk-analysis instrument that could 
be used in risk management decision making for regulatory, legislative, and 
enforcement activities (spring 2006). In the interim, Transport Canada will 
continue to rely on its inspectors and existing analytical capacity to address 
emerging security needs. 


The security inspection program has been expanded 


2.30 We reviewed how the security inspection system operates. We 
interviewed senior Transport security managers at five major airports and at’ 
national headquarters about the staffing of security inspection positions, 
training of security inspectors, and local ability to implement the inspection 
cycle called for by Transport policy. We reviewed organization charts and 
staffing plans for the airports we visited. We also reviewed headquarters’ 
quality control program for security inspection. 


2.31 Our review did not identify any major difficulty in local capacities to 
carry out the number of inspections required by headquarters. Positions were 
nearly all filled, and managers did not report any difficulty recruiting and 
retaining inspectors. Managers said the training program was good, a major 
improvement over the previous one. 


2.32 We noted that the annual Comprehensive Security Review called for 
by headquarters since April 2004 had not yet been carried out. Transport 
officials said that several comprehensive reviews were conducted in 2001, one 
just after the September 11 attacks. Officials were concentrating on 
expanding the inspection service in 2004-05. The new inspection policy will 
come into effect fully in 2005-06. 


2.33 ‘Transport Canada established a three-person Quality Review division 
in February 2003 but we found that it had not yet done any quality reviews, 
though officials told us that in 2004-05 they were intending to complete two 
quality reviews in Toronto and Montréal. Failure to conduct a quality review 
of security inspections carries a risk that standards will not be enforced 
uniformly, with poor inspection quality as a result. Our review of inspection 
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files in the airports we visited found several inspections that had never been 
properly documented and for which there was no evidence of completion. 


Problems in enforcing corrective action 


2.34 ‘Transport Canada’s security inspection system is the primary means for 
security managers to know whether the security system is meeting the 
regulated standards. We therefore assessed the Department's ability to 
analyze breaches of security and enforce corrective action when systemic 
problems become evident. 


235 Analysis of security breaches. We assessed the quality of the data in 
the main inspection database (called SEMIS) to determine whether we could 
rely on it in our audit. We examined 110 files selected judgmentally at the five 
airports we visited. We also asked the senior Transport Canada security 
manager at each airport to comment on the accuracy of summary statistics 
generated by the SEMIS system. 


2.36 We found that SEMIS data were completely accurate in only 

60 percent of the files we reviewed. The most common error was misstating 
whether the investigation had been closed or not. Transport security 
managers at the airports we visited did not regard SEMIS as accurate, so they 
maintained their own local databases and files. 


2.37 ‘Transport Canada managers at headquarters acknowledged that the 
SEMIS database contained errors. Headquarters managers said they intend to 
clean it up but have not been allocated funds to do it. 


2.38 Given the problems with SEMIS, Transport Canada replaced it in 
April 2004 with a new system called SEPIRS. The new system is intended to 
address the shortcomings of the old system, but data from before 2004 require 
validation to determine the extent of their inaccuracy. 


2.39 We also asked Transport Canada security managers how they assessed 
risks and identified trends in non-compliance. At most locations, they use 
data from their own databases and perform their own analysis. Several 
complained that they had no access to data from other regions and therefore 
could not identify trends by specific carriers who operate nationally. Their 
main means of assessing trends in risks and non-compliance are the weekly 
conference calls by the senior security managers across the country and the 
monthly reports prepared by the Quality Review division. 


240 The lack of an accurate central database has hampered Transport 
Canada’s ability to assess emerging trends. Officials told us that the new 
SEPIRS system will allow monthly reports to be seen by all the regions. 


241 Problems in enforcement. A security system must be able to not only 
recognize risk and non-compliance but also enforce timely corrective action. 
We therefore looked for issues that affected the overall performance of the 
enforcement system. 


242 Security of passenger air travel is based on a number of security 
measures that include the gathering of intelligence, law enforcement by 
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police, questioning by ticket agents, primary screening of passengers, 
secondary screening of some passengers, use of protective officers on some 
flights, and fortification of the flight decks of aircraft. 


243 ‘Transport Canada has not analyzed the overall effectiveness of its 
security systems; it has only one security performance measure in place: the 
“infiltration failure rate” of passenger screening. This is the rate at which 
screeners fail to detect “threat objects” such as a simulated knife, simulated 
gun, or simulated explosive device when the inspectors try to carry them 
through passengers screening. However, the Department has chosen to 
classify that measure and the related information; accordingly, we may not 
report it. 


2.44. When Transport Canada’s inspectors encounter a failure in CATSA’s 
procedures, including an infiltration failure, an enforcement letter is issued to 
CATSA. Transport Canada has issued such letters to CATSA for failures to 


meet security standards. Examples include 
* incidents where unauthorized persons gained access to secure areas; 


* cases where there were failures to detect unauthorized objects with 
metal detectors; 


* instances where employees did not keep adequate logs, communicated 
ineffectively, performed poorly, or lacked valid certifications or 
designations; and 


* cases where screeners failed to conduct random tests properly. Transport 
Canada later issued an exemption to its regulations to recognize the 
difficulty of complying with the requirement for random testing while 
improving the screening process at the same time. 


245 In Transport Canada’s inspections of CATSA that were unrelated to 
infiltration tests, only three to four percent identified a deficiency that 
resulted in an enforcement letter. CATSA officials told us they considered 
some of those deficiencies to be minor. However, Transport Canada does not 
categorize its enforcement letters by importance. 


2.46 Neither Transport Canada nor CATSA adequately tracked action 
taken in response to individual enforcement letters. Neither organization had 
a complete and accurate inventory, and the numbers of letters on file at 
Transport, CATSA, and in the Transport database did not agree. Neither 
agency could find responses to all the enforcement letters. We could not find 
responses to about 12 percent of enforcement letters related to infiltration 
tests and to 16 percent of letters addressing other deficiencies. CATSA 
officials did point out that they had changed procedures significantly in 
response to inspection results. During the audit, CATSA also began to track 
enforcement letters and its responses to them more closely. 


2.41 Aspects of Transport Canada’s enforcement model seem unsuited to 
dealing with a government agency that, like CATSA, is the sole provider of a 
security service. An air carrier or airport authority that showed persistent 
non-compliance would eventually be fined. However, Transport officials did 
not think monetary penalties against CATSA would be appropriate since, in 
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their view, fines would inevitably redirect resources away from the delivery of 
CATSA’s mandate. Further, because fines for security breaches are not 


disclosed, the officials felt that monetary penalties would have a limited 
impact on CATSA. 


248 ‘Transport officials told us they do not have a major problem with 
CATSA. Nevertheless, we found the following: 


* Transport Canada has never established any system-wide performance 
standards for CATSA that would rate as satisfactory anything less than 
100 percent compliance. 


- Transport security officials expressed dissatisfaction with CATSA’s 
performance in specific cases and have threatened to fine the agency. 


+ Transport Canada has never completed an assessment of CATSA’s 
overall performance to determine whether additional enforcement 
measures are required. 


* Transport staff were not aware of trends or patterns in the effects of 
enforcement actions. 


249 We do not believe that Transport Canada’s enforcement regime works 
well in the case of CATSA. No performance goals have been established. The 
Department’s internal studies and analyses are not adequate, and there 
appear to be no effective sanctions available should education and 
encouragement fail. 


250 Recommendation. Transport Canada should put in place system-wide 
performance measures that specify what it considers to be satisfactory 
performance by the Canadian Air Transport Security Authority (CATSA). 


Transport Canada’s response. Transport Canada has developed a 
comprehensive enforcement program, including monitoring and inspection, 
to ensure compliance with the rules. With respect to CATSA, Transport 
Canada has taken, and will continue to take, an incremental approach to 
enforcement. While Transport Canada does not consider monetary penalties 
to be necessarily the best compliance tool, it can take appropriate 
enforcement action, drawing if necessary on the range of legislative and 
administrative mechanisms available, including holding directors and officers 
accountable. In addition, Transport Canada is now developing system-wide 
qualitative and quantitative measures of performance by screeners and 
equipment, which will be reviewed on a regular basis. Measures will be 
finalized by early 2006. 


Installation of explosives detection systems is going well 


251 The 2001 Budget allocated over $1 billion to purchase, deploy, and 
operate advanced explosives detection systems (EDS) at airports across the 
country. These systems are to be in place before 1 January 2006. 


252 Pre-boarding screening of passengers and the effects they will carry on 
board includes physical searches and machines that can detect minute traces 
of explosives residue. 
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2.53 “Hold-bag” screening is the security screening of luggage that 
passengers have checked to be stowed in the hold (cargo compartment) of 
the plane. Screening officers identify threat objects by using enhanced X-ray 
or CAT-scan machines and other explosives trace equipment or by physically 
searching the bag. 


2.54 Part of CATSA’s screening responsibility includes acquiring, installing, 
and maintaining explosives detection equipment at airports and ensuring 
compliance with the regulations and standards Transport Canada has 
established for EDS. Transport Canada monitors operations and equipment 
for passenger and hold-bag screening to ensure that the systems meet its 
standards. 


2.55 Our audit looked at whether Transport Canada and CATSA each 
managed its part adequately in acquiring and implementing explosives 
detection systems. We reviewed five Class I airports (Vancouver, Montreal, 
Calgary, Ottawa, and the new Terminal | in Toronto); two Class II airports 
(Saskatoon and St. John’s); and two Class “other” airports (Sydney and 


Lethbridge). 


256 CATSA was to equip 89 airports across Canada for hold-baggage 
screening. It defined projects in these 89 airports and had completed about 
half of them at the time of our audit. CATSA reviewed the proposals from 
airports against its guidelines, proposed alternative solutions, and reached 
agreement with the airports on the installation of EDS. The airports were 
responsible for project construction and for installing the systems, and 
CATSA reimbursed them for their approved costs. We observed that project 
objectives were clearly defined, project implementation reports were given to 
CATSA’s Board of Directors, and there was an information system to track 
whether projects were on schedule and to track the costs. 


257 CATSA’s capital budget was based on completing the defined projects 
but did not include EDS at the domestic portion of a major airport, which 
would be undergoing renovations first. Transport Canada told us that to 
protect vulnerabilities in the meantime, it will ensure a level of security 
equivalent to that in the rest of the transportation system. Proceeding in this 
way appears to be reasonable, avoiding the expense of putting new systems in 
place that would have to be removed during renovations and then reinstalled. 
However, the size of the financial obligation this approach creates for CATSA 
is not clear. 


2.58 We expected that Transport Canada would have adequately defined 
the need for the EDS project and analyzed options. We found that it had 
identified the need for new EDS machines and considered EDS approaches, 
had done some of the research, and had analyzed options and chosen 
equipment well before September 11, 2001. It completed another options 


analysis after September 11. 


259 We noted a lack of life-cycle-costing information and analysis by 
Transport Canada, and CATSA only recently began to analyze the costs of 
explosives detection equipment over its life cycle. While the choice of 
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Marine security 


equipment was therefore based on incomplete information, we believe that 
given the emergency, the decisions were reasonable. 


260 At the time of the 2001 Budget, the government did not know exactly 
which aspects of marine security needed to be addressed and in what order of 
priority. As an immediate response, the Budget 2001 initiative identified a 
few “quick fix” improvements such as increased aerial surveillance of 
Canada’s maritime zone. An initial investment of $60 million was allocated to 
marine security in the knowledge that a more comprehensive approach would 
be needed. 


261 After about $25 million of this amount had been allocated to specific 
projects, an Interdepartmental Marine Security Working Group of senior 
officials conducted threat assessments, developed options, and made 
recommendations to ministers. The government responded by allocating an 
additional $172.5 million to fund improvements in marine security from 2003 


to 2008. 


262 This new funding is designed to improve 


* awareness of what is happening in Canada’s coastal waters (surveillance 
or domain awareness); 


* security of port facilities and waterways; 
* response time to events that threaten security; and 


* rates of inspection of vessels, ports, and cargo containers. 


263 Furthermore, the International Maritime Organization (IMO) 
developed international standards for the security of ships and port facilities. 
As a signatory to conventions under the IMO treaty, Canada became 
responsible for implementing the Automated Identification System (to 
identify vessels) and the International Ship and Port Facility Security Code. 
Our audit looked at the implementation of those two projects and a third, the 
High Frequency Surface Wave Radar installation. 


Automatic Identification System for ships is being implemented 


2.64 The International Maritime Organization requires that all ships of a 
certain size be fitted with an automatic identification system (AIS) that has 
transponders similar to those on aircraft. An AIS transponder broadcasts a 
message that identifies the vessel, its heading, and its speed. This continuous 
broadcast can be received by vessels, shore stations, or aircraft outfitted with 
AIS receivers. The completion of the Canadian Coast Guard’s AIS shore- 
based receivers is expected to enhance Canada’s ability to identify and track 
AIS-equipped vessels in its coastal areas. 


265 The Canadian Coast Guard is installing equipment on shore to allow 
the tracking of AIS-equipped vessels up to 40 nautical miles from shore, 

at an estimated cost of $27.5 million over five years. The St. Lawrence 
Seaway Management Corporation has already implemented this system in 
the St. Lawrence River. 
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2.66 We have two observations. First, a threat and risk analysis by the 
marine security Working Group identified four priority areas, but the 
allocated funding was enough for only the top three; we did not see a plan for 
addressing the shortfall. Second, setting up a project management IT system 
and obtaining funding approvals encountered delays, but the Coast Guard 
intends to complete the project by its original implementation date. 


2.67 Canada is a signatory to the International Convention for the Safety of 
Life at Sea (SOLAS); the new International Ship and Port Facility Security 
Code (ISPS); and amendments to both, adopted in December 2002 at the 
Diplomatic Conference of the International Marine Organization. Canada 
was required to implement, by | July 2004, the domestic ship and marine 
facility security requirements stipulated in the ISPS Code. 


2.68 ‘Transport Canada’s implementation of the ISPS Code was achieved by 
incorporating it in the Marine Transportation Security Regulations under the 
Marine Transportation Security Act. 


269 The Code and the Regulations apply to commercial vessels that weigh 
500 tons (gross tonnage) or more and to marine facilities and ports that serve 
such vessels. Regulations require that these vessels and facilities complete 
security assessments and security plans and designate security officers. 


270 Because Canada wants to harmonize its marine transportation 
regulations with those of the United States, it extended its regulations beyond 
the IMO requirements to also cover 


* cargo vessels of 100 tons (gross tonnage) or more; 
¢ vessels that tow barges carrying certain dangerous cargoes; 
* vessels carrying more than 12 passengers; and 


* marine facilities and ports that serve these vessels. 


271 We expected Transport Canada to ensure that the design of the 
national framework for implementing the ISPS Code in Canada would 
comply with international requirements. We further expected it to ensure 
that Canadian ship owners and port facility operators would implement the 
national framework to meet international requirements and the regulations 
under the Marine Transportation Security Act. 


2.72 We found that Transport Canada’s regional offices and marine 
inspection officers maintained good relations with ships and port facility 
operators during the process of issuing a security certificate. Where security 
plans required additional preparation or the formal appointment of a ship or 
port facility security officer, interim security certificates were issued. Future 
demand for renewals of security certificates could place considerable pressure 
on Transport Canada’s marine security inspectors to complete the required 
inspections before the renewal deadline. 


273 The ISPS Code includes standards to classify and increase security 
levels in response to threats, as well as procedures to be followed until 
information becomes available that justifies a return to lower levels. We found 
that the criteria used by Transport Canada to increase marine security levels 


are not defined clearly. 
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Federal emergency preparedness 


2.74 Overall, Transport Canada successfully completed the necessary 
procedures for issuing security certificates to all ship and port facility 
operators required to have them, and it met the ISPS Code Agreement 
deadline of 1 July 2004. . 


High Frequency Surface Wave Radar faces funding and regulatory problems 


275 Onthe basis of its threat and risk analysis, the marine security Working 
Group supported the development of High Frequency Surface Wave Radar 
(HFSWR) that, as a long-range, over-the-horizon radar, can provide real- 
time, continuous radar surveillance extending up to 200 nautical miles. 


2.76 National Defence has worked for 15 years on a research project to 
develop radar effective over water to a distance of 200 nautical miles. The 
readiness trial was completed in August 2004, and the Department has 
informed us that system-acceptance tests were completed recently. The 
results, while promising, have not yet been analyzed completely. 


2.71 We examined this initiative as a capital project, assessing the 
* needs definition, 
* options analysis, and 


* project management and definition. 


278 We note two concerns about the ongoing development of the HFSWR 
project: 


« A National Defence study indicates that full coverage would cost a total 
of $220 million rather than the $43.1 million the Department was 
allocated. National Defence recommended acquiring and installing 
equipment at only four to six sites. Research has shown that HFSWR 
does perform to expectations during normal circumstances, but it does 
not operate to its full range under certain conditions such as at night, 
during meteorological disturbances, and in heavy seas. Increased costs 
and reduced capability leave major gaps in the coverage of Canada’s 
coastlines. However, National Defence believes that this radar provides 
additional information not available from existing surveillance systems. 
We were unable to obtain evidence that explains how gaps in coverage 
would be closed, the cost of closing them by other means, or the security 
implications of leaving them unaddressed. 


* National Defence has not yet obtained a permanent licence from 
Industry Canada to operate the system. A permanent licence can be 
issued only after Industry Canada is assured that the system will not 
interfere with the operations of other licensed users of the same radio- 
frequency band. 


279 The front line of defence in a terrorist attack is the first responders— 
the police officers, firefighters, emergency medical care providers, and 
emergency management officials who make up specially trained hazardous 
materials teams, urban search and rescue units, bomb squads, and tactical 
units. Most of them work for a provincial, territorial, or municipal 
government. 
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2.80 The National Security Policy of April 2004 says that first responders 
are at the heart of the emergency management system; the federal 
government “will often play only a supporting role in emergency management 
to provinces and territories, communities and the private sector.” The Policy 
identified the need to modernize the national system of emergency 
management. 


281 On September 11, 2001, Canada’s approach to emergencies was based 
on a Cold War approach. The system was based on a highly decentralized and 
distributed division of responsibilities among front line responders, provinces 
and territories, and lead departments at the federal level. Although the 

2001 Budget had allocated significant additional funds to improving 
emergency preparedness, by 2003 the government recognized that 
organizational improvements in this area would be needed to make the 
desired progress. It therefore removed the Office of Critical Infrastructure 
Protection and Emergency Preparedness (OCIPEP) from National Defence 
and merged it with the Solicitor General and the National Crime Prevention 
Centre to form Public Safety and Emergency Preparedness Canada 
(PSEPC).This new department is intended to improve the integration of 
government efforts across the public safety program, linking the various 
federal programs together more closely. 


2.82 In its first year of operation, PSEPC initiated a major legislative 
program to give substance to the government’s goal of “a seamless national 
emergency management system.” The Department has also launched the 
Government Operations Centre, begun the development of the National © 
Emergency Response System (NERS), and improved the readiness and 
exercise program. We report here on the progress made and work still to be 
done below. Many of the gaps and implementation problems our audit found 
originated in the program before the new department was established. 


283 The 2001 Budget allocated $513 million over five years to federal and 
national efforts to prepare for chemical, biological, radiological, and nuclear 
threats, or CBRN (Exhibit 2.3). The funds were to enhance existing 
laboratory networks so they could more quickly detect and identify 
biochemical threats. The government also decided to buy new protective 
equipment for emergency response teams and improve their training. 


284 In this audit, we looked at whether federal programs to enhance 
emergency preparedness for a CBRN event have been managed adequately. 
We also looked at critical infrastructure protection. Budget 2001 had 
allocated $396 million to enhancing the management of critical 
infrastructure protection. We looked at whether threat and risk assessments 
had been used to prioritize investments in infrastructure and at the progress 
achieved to date. 


285 We did not examine progress made by provincial, territorial, or 
municipal governments. Our findings relate solely to the federal government. 
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Exhibit 2.3 Chemical, biological, radiological, and nuclear events 


The federal government has recognized that Canada is not immune from the threat of 
terrorism following September 11, 2001, and there is growing concern over the threat 
of chemical, biological, radiological, or nuclear (CBRN) terrorism. 


Potential CBRN weapons encompass a range of agents: 
* biological (such as smallpox and anthrax) 
¢ chemical (such as sarin gas disseminated by explosives or aerosols) 


¢ radiological/nuclear (such as radioactive material scattered by conventional 
explosives) 


As with other types of emergencies, responsibility for CBRN-incident response is 
shared by federal, provincial, and municipal governments. Civil emergencies are 
initially dealt with by first responders—police, firefighters, and emergency medical 
personnel. If additional assistance is required, local officials contact the province or 
territory, who in turn can seek assistance from the federal government. 


Source: Government of Canada, February 2003 


286 ‘To examine emergency preparedness for a CBRN incident, we looked 
at three areas: 


¢ chain of command—that is, who is in charge of the response to an 
incident; 


* federal and national capacity to respond to an incident (national 
capacity being the combined resources of federal, provincial/territorial, 
and municipal agencies); and 


* the testing and exercising of response plans. 


The chain of command remains complex 


2.87. We expected to find a clearly established chain of command to guide 
the federal response to a CBRN incident. This would mean that federal roles 
and responsibilities would be clearly defined and would link consistently with 
response plans of provinces and territories. 


288 Canada’s approach to managing emergencies has been based on a 
division of responsibilities among first-line responders at the municipal and 
provincial/territorial levels and the lead departments at the federal level. 


289 The National Security Policy stated the problem clearly: 


National emergency co-ordination currently suffers both from 
the absence of an effective federal-provincial-territorial 
governance regime and from the absence of commonly agreed 
standards and priorities for the national emergency 
management system. 


The government is taking steps to rectify these issues, but we found that they 
go only partway to establishing a clear command and control structure. 
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290 Before December 2003, the nature of an emergency determined which 
federal department had the lead role in managing the response. Health 
Canada would manage a biological, radiological, or nuclear emergency and 
‘Transport Canada the cleanup of a transportation accident involving 
biological, chemical, or radiological materials. Natural disasters would be 
managed by the Office of Critical Infrastructure Protection and Emergency 
Preparedness (OCIPEP) and incidents affecting the food chain by the 
Canadian Food inspection Agency. If any of these incidents proved to be an 
act of terrorism, leadership would immediately shift to the Department of the 
Solicitor General. 


291 December 2003 was when, as already noted, OCIPEP was moved out 
of National Defence and merged with the Department of the Solicitor 
General to create Public Safety and Emergency Preparedness Canada 


(PSEEC): 


292 As part of creating PSEPC, the government put forward legislation 
(Bill C-6) that would provide for the leadership role of the Minister of Public 
Safety in the event of a national emergency. The Minister would be 
responsible for providing strategic co-ordination while respecting the Prime 
Minister’s prerogative in matters relating to national security and the 
statutory authorities of other ministers. According to officials, however, 

Bill C-6 does not provide the specific powers needed to activate this role. 
Those powers are expected to be contained in revisions to the Emergency 
Preparedness Act on which PSEPC plans to begin consultations in the 

spring of 2005. 


2.93 Recommendation. Public Safety and Emergency Preparedness Canada 
should finish drafting the revisions to the Emergency Preparedness Act as soon 
as possible to finalize the definition of the minister’s powers and 
responsibilities. 


Public Safety and Emergency Preparedness Canada’s response. The 
modernization of the Emergency Preparedness Act is a commitment under 
the National Security Policy. A discussion paper has been drafted outlining 
the proposed powers and authorities of the Minister of Public Safety and 
Emergency Preparedness. The paper reflects the emerging requirements of 
emergency management. Consultations with stakeholders will begin in 


spring 2005. 


294 Anew national response system. PSEPC is developing a new national 
emergency response system (NERS) aimed at eliminating overlap and 
duplication in the crisis- and consequence-management stages of response 
(Exhibit 2.4). It intends to standardize all federal response plans and their 
links to provincial plans, but it has not committed to a completion date. We 
were told that the National Counter-Terrorism Plan and the National 
Support Plan will be revised and become part of the NERS, but currently the 
NERS is not in a form that could replace an existing federal plan. 
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Exhibit 2.4 National Emergency Response System (NERS) 


NERS is an “all hazards” response structure designed to co-ordinate the federal 
response to emergencies of national importance, including support to provinces, 
territories, and the federal departments involved. It will provide for national policy 
direction and strategic co-ordination. 


A national response within NERS may also include issuing a notification, a warning, 
and/or an alert to the public. 


NERS will not change existing departmental mandates but rather will co-ordinate 
mandates in a harmonized federal response. 


NERS is based on functionally oriented groups staffed by personnel from Public Safety 
and Emergency Preparedness Canada and other federal departments. The response 
structure can be activated partially or fully, using escalating levels of operations that 
correspond to emerging, imminent, or occurring emergencies. Each response level 
represents an increase in the federal capability and capacity to respond. 


Source: Public Safety and Emergency Preparedness Canada 


2.95 | PSEPC has conducted extensive consultation on the NERS within the 
federal government and with the provinces and territories. We were told that 
the government has approved the approach in principle. Officials have also 
told us that they are developing the new system’s structure and hope to get 
formal Cabinet approval sometime in spring 2005. In our opinion, if NERS is 
to work it must have official sanction and, equally important, support from all 
federal agencies involved in responding to a national emergency—which, by 
nature, can involve several agencies. Until the government fully implements 
the NERS, elements of the federal response will still be managed by individual 
departments. 


296 Recommendation. Public Safety and Emergency Preparedness Canada 
should work with the other federal agencies to clarify the command and 
control structure governing the federal response to emergencies. 


Public Safety and Emergency Preparedness Canada’s response. Leadership 
will be exercised through the command and control structure of the National 
Emergency Response System (NERS). In addition, changes to the Emergency 
Preparedness Act will reinforce the authority of the Minister of Public Safety 
and Emergency Preparedness to co-ordinate the actions of all federal players 
in emergencies of national significance. 


2.97 Recommendation. Public Safety And Emergency Preparedness 
Canada should obtain its federal partners’ formal agreement to the National 
Emergency Response System as soon as possible. 


Public Safety and Emergency Preparedness Canada’s response. We agree 
with the recommendation. A formal agreement to adopt the National 
Emergency Response System (NERS) will be sought with federal partners. 
There will be a good opportunity to validate the NERS through Exercise 
Triple Play in April 2005. In addition, provincial and territorial partners have 
agreed with the approach taken by the federal government. 
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298 Co-ordinating operations centres. We identified 27 operations 
centers in departments across the federal government whose operating 
capacities and technologies vary. We noted a lack of integration among these 
centres. In September 2001, OCIPEP did not have secure communications 
with these federal operations centres or with those of Canada’s allies. 


2.99 The National Security Policy of April 2004 announced the creation of a 
new Government Operations Centre in Public Safety and Emergency 
Preparedness Canada (PSEPC) to manage the federal response to emergencies. 
As of September 2004, the Centre has secure communications with other 
federal operations centres and with the United States and the United 
Kingdom. Co-ordination between individual departments’ operations centres 
and the Government Operations Centre is an important element of the NERS. 


2.100 The Centre’s basic monitoring activities began in September 2004, but 
its emergency response capacity has not been validated in an exercise. It does 
not have command and control authority over the federal response when the 
lead resides in another federal department. In anticipation of the legislative 
changes, PSEPC officials are developing criteria for use of the Centre by other 
government departments as well as guidelines for transferring the federal lead 
to PSEPC when that is considered to be in the national interest. We note that 
the Government Operations Centre (GOC) is a critical component of the 
NERS concept of operations, which calls for the GOC to be proactive in 
conditions of emergency or uncertainty. Each of the response levels provides 
additional resources to ensure that the GOC has the capability and capacity 
to co-ordinate timely readiness and response activities. In practical terms, as 
the response level increases, the staffing and resource levels of the various 
functional groups will also increase. 


Integrated plans are essential to an effective response 


2.101 Another important element of the NERS is the link between national 
and departmental emergency plans. 


2.102 Managing the federal response to a CBRN event could draw on a 
number of emergency plans. For example, we reviewed the following: 


¢ National Counter-Terrorism Plan 


National Support Plan 


Federal Nuclear Emergency Plan 


Food and Agriculture Emergency Response System 


2.103 We found that departmental plans are vague on how they would link 
together to form a co-ordinated federal response. Section 7 of the Emergency 
Preparedness Act requires that departments prepare emergency response plans 
for areas within their mandates. We noted a potential, as emergencies develop 
and implicate more departments and agencies, for conflict between having to 
work together with other departments and supporting the line responsibilities 
of their own mandates. Officials at PSEPC could not show us an inventory of 
departmental emergency response plans that could be activated in the event 
of a CBRN terrorist event. Nor could they provide us with an analysis 
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showing how, in a complex emergency involving several departments, the 
plans would work together to achieve a seamless federal response. 


2104 ‘To achieve a national response, federal response programs need to be 
integrated not only across departments but also with those of other 
jurisdictions. We expected to find that federal response plans would link 
across jurisdictions consistently. However, we found the National Counter- 
Terrorism Plan and the National Support Plan to be inconsistent in the level 
of detail and the depth to which they link with response plans of the 
provinces. 


2.105 Recommendation. Public Safety and Emergency Preparedness Canada 
should work with its federal partners and the provinces and territories to 
improve the co-ordination of response plans. 


Public Safety and Emergency Preparedness Canada’s response. We agree 
with the recommendation. Work has started on improving the co-ordination 
of response plans with provinces and territories. This initiative is also one of 
the action items flowing from the Federal-Provincial-Territorial Ministerial 
Meeting on Emergency Management, held on 24 January 2005. 


Building capacity to respond to terrorist events 


2.106 ‘Io improve the national capacity to respond to CBRN events, the 
federal government chose two areas of action: enhancing the capacity of 
federal agencies to respond, and increasing the capacity of provincial and 
municipal first responders. 


2.107 The $513 million that Budget 2001 provided to enhance CBRN 
equipment and response capacity was allocated to a number of federal 
departments. The CBRN Research and Technology Initiative was created to 
distribute some of the funding and to allocate research and development 
projects. The RCMP Canadian Forces, Health Canada, and Transport 
Canada were specifically tasked with developing a CBRN response capability. 


2.108 Risk management principles acknowledge that although risk cannot be 
eliminated, enhancing protection from existing or potential threats can help 
reduce it. Accordingly, a risk management approach is a systematic process to 
analyze threats, vulnerabilities, and the criticality (or relative importance) of 
assets to better support key decisions. 


2.109 We expected to see that programs designed to enhance capacity had 
been based on risk analyses and scenarios setting out what could happen, how 
departments should respond, and what resources the responses would require. 
Threat and risk assessments are key tools to determine where and how to 
establish programs. 

2.110 The RCMP is increasing capacity. The RCMP was allocated 

$23 million over five years to equip and train first responders in its force and 
to develop specialized response teams. We found that it prioritized its capacity 
investments according to threat and risk analysis. 

2.111 The RCMP developed a CBRN doctrine and concept of operations as a 
basis for its equipment purchases and training. It has now equipped and 
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trained 3,600 members to a basic response level and is scheduled to equip and 
train all 15,000 members of its regular force by April 2006. CBRN response 
training is now included in its training curriculum and, effective 

October 2004, all new graduates have received this training. 


2.112 The RCMP has also equipped and trained four specialized two-member 
CBRN emergency response teams that support local first responders. Its plan 
to equip and train such teams was based on threat and risk assessments. The 
original assessment of the need for a national response capacity determined 
that eight teams of six to eight members each were needed. However, we were 
told that funding was provided only for a smaller response capacity. 


2.113 The teams are equipped with CBRN-capable bomb suits; protective 
suits; detection, mitigation, and neutralization devices; and the capability to 
conduct forensic scene examinations in a CBRN environment. However, the 
small number and size of the teams and the large geographic areas to be 
covered could significantly delay the response in some regions. The RCMP 
has not yet determined how it will meet the identified need. 


2.114. Recommendation. The RCMP should improve its capacity to respond 
to chemical, biological, radiological, and nuclear emergencies. 


RCMP’s response. The RCMP originally estimated the need for highly 
specialized CBRN response teams to be deployed regionally across Canada to 
supplement other response capacities. At the same time, the RCMP identified 
various other CBRN response requirements. 


Since that time, we have made extensive progress toward training and 
equipping all RCMP members as CBRN first responders, able to assess and 
undertake primary containment of possible CBRN incidents. 


The specialized Regional Response Teams are located in the Pacific, North 
West, Central, and Atlantic regions. They operate in conjunction with front- 


line responders and the Joint National RCMP/DND CBRN Response Team 


to provide a measured response in all parts of Canada. 


The RCMP will assess the optimum size of the Regional Response Teams, 
taking into account current levels of training, including with partner 
agencies, as well as integration, new technologies, and availability of 


resources. 


2115 National Defence has a limited role. In 2002, the National CBRN 
Response Team was established, combining the resources of three federal 
departments. The RCMP has the overall lead and handles CBRN explosive 
ordinance and forensic evidence. The Public Health Agency of Canada 
(PHAC, created in September 2004 to assume some of Health Canada’s 
responsibilities) provides emergency medical teams and a transportable lab to 
deal with biological hazards. National Defence provides its Joint Nuclear, 
Biological and Chemical Defence Company, whose primary mission is to 
support the RCMP the first-responder community, and civil authorities with a 
dedicated military capability to respond to CBRN terrorist activities. 

Exhibit 2.5 provides more detail on the role of the Canadian Forces. 
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Exhibit 2.5 The Role of the Canadian Forces 


While the entire Canadian Forces is potentially available to provide support, its primary 
role is not that of a first responder. The Forces can provide general support to provinces 
and territories to deal with the consequences of a chemical, biological, radiological, or 
nuclear event at their request but are not specifically tasked with this role. In 
December 2003, the government announced “an increase in the Canadian Forces 
Reserves available for civil preparedness, including a capacity to deal with natural 
disasters and local emergencies.” The Reserves are not currently equipped and trained 
for this capacity, but National Defence is studying the feasibility of using them. 


Source: National Defence 


2.116 Budget 2001 allocated $30 million to National Defence to establish a 
new military unit, the Joint Nuclear, Biological and Chemical Defence 
Company. This military company can be used to support other government 
departments and the provinces and territories in their responses to a CBRN 
terrorist event. An evaluation by National Defence found “serious command 
and control issues” with the new unit. 


2.117 Emergency medical teams have not been created. In 
December 2001, Health Canada was allocated $501,000 to develop Health 
Emergency Response Teams 


rapid response teams of physicians, nurses, and 
medical technicians. During emergencies, these federally 

trained medical teams are to come to the support of local health care workers. 
A national office was established and given a broad mandate to oversee 
funding, recruitment, equipping, and deployment of the teams. In 

January 2003, Health Canada was allocated $626,000 annually to train 
health care workers in the prevention and treatment of smallpox. As part of 
this funding, the Department was to train provincial Smallpox Emergency 
Teams to immunize the public in the event of a smallpox epidemic. At the 
time of our audit, neither Health Emergency Response Teams nor the 
Smallpox Emergency Teams had been established. In both cases, officials from 
Health Canada now with PHAC told us that there are legal issues that have 
yet to be resolved. 


2.118 Health Canada and the Public Health Agency of Canada are 
mandated to support the provinces and territories in managing the public 
health aspects of an emergency. A Budget 2001 allocation of $88.6 million 
over five years is targeted to strengthening their public security and anti- 
terrorism response capacity. We expected that investments would be 
prioritized on the basis of risk assessments and casualty scenarios, but we 
found limited use of such analyses. 


2.119 PHAC manages the National Emergency Stockpile System (NESS) 
previously managed by Health Canada. The NESS is a cache of medical 
equipment and pharmaceuticals contained in warehouses across the country 
and ready to be sent to provinces and territories at their request. Budget 2001 
allocated $7.9 million to replenishing this stockpile. We expected that the 
contents would be based on a risk assessment and that plans would be in 
place to ensure timely, efficient distribution. 
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2.120 It is not clear what role the stockpile plays in the emergency 
management system. Neither its objectives nor its contents are managed under 
a federal emergency response plan. The contents of the stockpile are not based 
ona risk assessment. The stockpile contains no CBRN equipment, such as 
protective gear, detectors, and decontamination showers and apparatus. It does 
contain a quantity of CBRN pharmaceuticals and medical countermeasures. 


2.121 During our audit, PHAC completed the first stage of a strategic 
analysis examining the role of the National Emergency Stockpile System. 


2.122 Transportation issues. A study looking at emergency transportation 
issues concluded that existing transportation arrangements for the NESS are 
“moderately reliable.” Parts of the national emergency stockpile were moved 
after September 11, 2001 to allow for their distribution anywhere within 

24 hours. Previously, the transportation and distribution of supplies in an 
emergency had been problematic and arranged on an ad hoc basis. For 
example, in response to the grounding of so many civilian aircraft on 
September 11, provincial officials requested supplies from the NESS. The 
only viable option for moving them was a military airlift. The plan was to load 
transport trucks with beds and blankets and send them to the Ottawa airport. 
Late on the night of September 11, it became clear that the trucks would 
have to be sent instead to CFB Trenton, because the Canadian Forces no 
longer kept equipment at Ottawa to load aircraft. A departmental after- 
action report concluded that these logistical problems resulted in a significant 
delay in shipping some of the supplies. Our view is that in an emergency, time 
can be a critical factor and it is not a good point at which to develop 
alternative distribution arrangements. 


2.123 We recognize that the complexity of moving NESS supplies efficiently 
in an emergency is affected by the distances involved, the mode of transport, 
and the need for specialized personnel. In past emergencies, NESS managers 
have relied on the military to airlift supplies. However, the military’s 
operational commitments can limit the availability of aircraft, so it cannot 
guarantee immediate transportation of NESS supplies. PHAC has not 
secured alternative airlift arrangements. 


2.124 One exercise we reviewed noted the risks involved in shipping medical 
countermeasures by commercial aircraft. Officials told us that they are 
developing an agreement with the RCMP to ship emergency drugs in its 
aircraft. 


2.125 We noted that as part of developing the National Emergency Response 
System, work is beginning on a National Emergency Transportation System 
(NETS). The NETS task is to address the need for a comprehensive 
transportation service within 24 hours. Public Safety and Emergency 
Preparedness Canada now has the lead in developing the NETS. 


2126 Recommendation. The Public Health Agency of Canada should 
co-ordinate the management of its National Emergency Stockpile System 
(NESS) with other federal agencies and include the NESS in the National 


Emergency Response System. 
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Public Health Agency of Canada’s response. PHAC is working closely with 
Public Safety and Emergency Preparedness Canada on a number of fronts, 
including input into the development of the National Emergency Response 
System (NERS) and a National Health Emergency Management System. Part 
of that process will include ensuring that the NESS is closely linked to the 
emergency response activities outlined in the NERS. 


2.127 Recommendation. Based on risk assessments and casualty scenarios, 
the Public Health Agency of Canada should update the contents of the 
national emergency stockpile as soon as possible. It should also secure 
arrangements for the transportation and distribution of supplies during 
emergencies. 


Public Health Agency of Canada’s response. PHAC is currently 
undertaking a strategic review of the NESS that includes a review of the 
contents of the stockpile. The strategic review, to be completed in 2006, will 
include risk assessments (initial risk assessment completed November 2004) 
and casualty scenarios. Emergency transportation of NESS supplies is being 
addressed through the National Emergency Transportation System (NETS). 


PHAC’ s Regional Emergency Response Co-ordinators will soon be 
developing NESS provincial and territorial emergency transportation plans in 
collaboration with each jurisdiction. 


Use of the Special Access Program is inappropriate 


2.128 Most CBRN medical countermeasures are not licensed for use in 
Canada; they are imported from the United States through the Special 
Access Program (SAP). For example, NESS managers use SAP to supply first 
responders with CBRN medical countermeasures. The Special Access 
Program was designed to give doctors access to drugs not licensed for sale in 
Canada, with control based on one doctor prescribing drugs for one patient. It 
was not designed for mass distribution of unlicensed drugs as medical 
countermeasures. 


2.129 Recommendation. Health Canada should establish an appropriate 
legal structure for providing unlicensed drugs in an emergency. 


Health Canada’s response. Agreed. Health Canada proposes to develop the 
regulatory authorities, in consultation with stakeholders, to enable the 
Minister to authorize a block release of products that have not been approved 
for sale in Canada for the purpose of responding to public health emergencies. 
As well, Health Canada proposes to develop new regulatory authorities 
needed to authorize "emergency use" products for the medical 


countermeasures context. 


Smallpox vaccine stockpile is still being completed 


2.130 Following Budget 2001, Health Canada was allocated up to $48 million 
to develop smallpox vaccines and plans for their use and to purchase vaccine- 
related supplies. The use of smallpox virus as a biological weapon was 
identified as a risk, and scenarios were developed that projected up to 
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10 million casualties. In March 2003, a contract was awarded for the 
procurement of smallpox vaccine. 


2.131 Due to problems in the manufacturing of the vaccine, Health Canada 
received only 5.7 million of the 10-million-dose requirement. It paid for only 
the doses it received. PHAC, now responsible for the stockpile, is not buying 
more vaccine to make up the shortfall; it is investigating the possibility of 
doing this by diluting the vaccine it has. At the time of our audit, it did not 
have in stock the 10 million doses identified as the required number. 


2.132. Under the smallpox vaccine program, Health Canada was to negotiate 
with the provinces by April 2004 on plans to administer the vaccine. At the 
end of our audit in December 2004, the plans had not been completed by 


PHAC. 


2.133 Recommendation. The Public Health Agency of Canada should 
determine how to meet its identified requirement for smallpox vaccine. 


Public Health Agency of Canada’s response. A decision on future 
procurement will be based on the results of the clinical trials of the current 
stockpile, a new risk assessment, and the results of the ongoing monitoring of 
smallpox vaccine research and development in 2006. In the interim, there is 
an adequate stockpile of smallpox vaccine and global commitment to assist in 
the event of an outbreak to ensure public safety and security. 


A research and technology initiative needs to assess capacity gaps 


2.134 The CBRN Research and Technology Initiative (CRTI) is a five-year, 
$170 million fund that Budget 2001 established to address CBRN threats. It 
involves 17 federal departments and agencies, led by Defence R&D Canada, 
an agency in National Defence. The CRT is mandated to strengthen 
Canada’s preparedness for, prevention of, and response to a CBRN attack. 
Funding (aside from $9.5 million for program administration) was divided 
among three main priorities: improving the technological capacity of federal 
laboratories ($27.5 million), accelerating the delivery of technology to first 
responders ($38 million), and investing in research and development 


($95 million). 


2.135 Forty federal laboratories participate in the initiative under a 
memorandum of understanding. We expected to find that the scheduling of 
investments in laboratory capacity had been guided by a risk assessment and 
gap analysis. 


2.136 We found that between January and March 2002, a threat and risk 
analysis by the CRTI identified gaps in the federal laboratory capacity. Three 
clusters or groupings of departmental laboratories were then established 
according to their chemical, biological, or radiological/nuclear capabilities. 
The laboratory clusters identified the technologies or capacities that each was 
to develop. The CRTI Steering Committee then approved funding proposals 


based on an agreed-upon risk assessment. 


2137 Of the $27.5 million in total funding allocated to acquiring technology 
in federal laboratories, the CRTI committed 77 percent in the first two years 
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to addressing the identified gaps. But how all the labs will work together and 
use this technology in an emergency has not yet been clearly defined. 


2.138 A Health Canada study of how the cluster of biological laboratories 
functioned during the September 2003 SARS crisis indicated that the 
clustering of laboratories improves the response to an emergency. The study 
report noted that more rapid analysis, and therefore more rapid prevention 
and mitigation measures, were possible because the cluster members involved 
in the incident were aware of the capacity and capabilities outside their 
organizations that they could call on to assist during the incident. We found 
that how the laboratories in the biological cluster and the chemical cluster 
will work together is still being developed. Cluster managers noted that the 
number of departmental response plans and the inconsistencies among them 
created problems for the laboratories in working together. They identified a 
tension between working together and supporting their operational 
mandates. 


2.139. The CRTI was also mandated to accelerate the delivery of technology 
to the first responder community. It developed a risk analysis known as the 
consolidated risk assessment to prioritize projects, and we found the analysis 
to be adequate. The consolidated risk assessment is updated every year to 
guide the annual selection of projects. 


2.140 By April 2004, the CRTT had funded 19 projects totalling $16.5 million 
to develop technology for first responders. It had also committed to funding 
22 research and development projects totalling $17.1 million. These projects 
typically follow a two- to four-year cycle, with completion of the first projects 
scheduled for November 2004. However, Budget 2001 did not include 
funding to acquire equipment based on the technology developed or 
accelerated by the CRTI. The $10 million allocated in Budget 2001 for the 
purchase of CBRN equipment was for only two years, before any CRTI 
projects would be completed. The CRTI is not designed or mandated for mass 
distribution of technology to first responders. 


Federal programs for first responders have progressed slowly 


2.141 Budget 2001 allocated funds to build a response capacity not only in 
federal departments but also at the local level. It set aside $59 million over 
five years for CBRN training of first responders, $20 million over five years to 
develop a national capability for heavy urban search and rescue, and 

$10 million over two years to buy CBRN detection and decontamination 
equipment and protective clothing for first responders. 


2.142 Consulting first responders. In late 2001, officials from OCIPER the 
Department of the Solicitor General, and National Defence met with first 
responders in eight Canadian cities. Their aim was to develop ways of 
strengthening the national capability to manage the consequences of terrorist 
attacks. The report that summarized these meetings noted the urgent need 
for a clear federal policy on enhancing the capacity of first responders to deal 
with a CBRN incident. It concluded that no jurisdiction had the capacity to 
deal with mass casualties. 
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2.143 The report stated that training was the most critical part of preparing 
for CBRN threats, and it was lacking across the country. All first responders, 
including health care professionals, needed basic training in CBRN 
awareness. Further, federal leadership was urgently needed to establish 
training standards or guidelines so that all those who received training would 
be trained to the same degree. Standardized training was seen as the way to 
ensure that different jurisdictions could co-ordinate an integrated response. 


2.144 First responders were said to be looking to the federal government to 
establish standards or guidelines for equipment as well. The report said it was 
clear that first responders were aware of the need to ensure that their 
equipment could interoperate with that of other jurisdictions when necessary. 


2.145 The report also said that the current arrangement for funding first 
responders’ CBRN equipment (including heavy urban search and rescue 
equipment) through the Joint Emergency Preparedness Program was too slow 
and cumbersome to be effective. It contended that a major overhaul of the 
program as it existed then was necessary for the timely provision of 
equipment. Furthermore, any funding model should take into account 
maintenance and longer-term replacement costs and allow for flexibility. 


2.146 We spoke with first responders primarily at the municipal level. We 
learned that they are well aware of the need to work together. First responders 
told us about new initiatives and co-operation agreements and protocols. 
They have developed new plans and revised others. In addition, PSEPC 
officials explained that they have held cross-country consultations aimed at: 
improving program delivery and have been working to improve the delivery of 


funds. 


2.147 Heavy urban search and rescue (HUSAR). Heavy urban search and 
rescue capacity is the location and removal of trapped persons in collapsed 
structures, using dogs and sophisticated search equipment. HUSAR capacity 
has increased slowly. 


2.148 Budget 2001 allocated $20 million over five years to develop a capacity 
for search and rescue in eight major cities. This investment was managed 
through an existing shared-cost program; the federal share was 75 percent. 
Under the Joint Emergency Preparedness Program (JEPP), the federal 
government co-funds provincial proposals. Shared funding of HUSAR 
equipment has proved to be problematic because of its high capital cost and 
the continuing costs to the municipalities of operating and maintaining it. 
Even a 75-percent federal share of the capital expenditure has proved 
insufficient to allow municipalities to take up the federal funds available. We 
found that at 1 April 2004, the provinces, territories, and municipalities had 
used less than 50 percent of the available funds. 


2.149 We expected to find funding targeted to high-risk communities and 
guidance provided to them on identifying the capacity they need. 


2.150 We found that the former OCIPEP had developed HUSAR team 
profiles and identified the needed equipment. Proposals by the cities for 
HUSAR funding were approved based on an analysis of the gap between 
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Surge capacity—Surge capacity refers to the 
ability of emergency response services to aid 
and support each other. Mutual aid can 
encompass the personnel, equipment, and 
resources need to provide relief to overwhelmed 
emergency services during a major incident. 
Interoperability and compatibility of protective 
equipment may be critical in large incidents, as 
in the case of respirators at the World Trade 
Centre in September 2001. Promoting the 
standardization of protective equipment would 
support equipment sharing in the field among 
response services. It would make the use of 
supplementary equipment delivered to a major 
event site (from federal caches, for example) 
possible as well, by assuring its compatibility 
with existing gear and requiring less training 
and fitting in the field. 


Source: Protecting Emergency Responders, Santa Monica, CA 
RAND Corporation, 2003 


existing and targeted capacities. Highest priority was assigned to five of the 
eight cities identified in the initial risk assessments. A sixth team is being 
developed, and the remaining two cities have expressed an interest in 
participating. 


2151 Providing CBRN equipment to first responders. OCIPEP used its 
existing Joint Emergency Preparedness Program (JEPP) to fund the purchase 
of $10 million in CBRN equipment for provincial and municipal first 
responders. It chose JEPP as the funding vehicle because, as an existing 
program, it could deliver funding quickly. The funding was for only two years, 
and the funds for the first year were received in the last quarter of the year. 


2.152 OCIPEP received proposals from all provinces and one territory; their 
costs would total more than the $10 million available. Since the federal 
response is to support provincial and local authorities, we expected to find a 
list of provincial and municipal resources and capabilities—information that 
would then guide the distribution of federal funding to develop local 
capacities in areas most at risk. The information could also be used to develop 
goals for national preparedness, along with measurable performance 
indicators. 


2.153 We did not find a risk analysis to guide funding decisions. We found 
that OCIPEP had not developed performance benchmarks for CBRN 
response teams or identified gaps in the response capacities of high-risk 
communities. Officials informed us that OCIPEP had relied on the expertise 
of provinces and territories to identify gaps in their respective jurisdictions. 
Rather than develop, fund, and equip a national first response capacity in a 
select number of cities as it did for HUSAR, OCIPEP spread the funding 


across regions, thereby diluting the response capacity. 


2.154 Halfway through the two-year program, the CBRN Steering 
Committee began using provincial population percentages to guide its 
allocation of funds. OCIPEP officials told us that this was meant to achieve 
proportionate distribution of the funds across the country. However, we found 
that in the first year, the funds had been allocated to the provinces that had 
submitted proposals first. Early allocations to provinces with low populations 
meant that funding by provincial population percentages was not achieved. 
Moreover, this approach was not based on risk. 


2.155 Although provincial populations were used to allocate funding, the 
funds were not then directed to the high-population centres in the provinces 
or to areas of highest risk. Across the country, cities with populations below 


300,000 received 40 percent of all the funds. 


2.156 Equipment guidance is lacking. We expected that to help first 
responders make informed choices in their purchase of off-the-shelf response 
equipment, the federal government would provide guidance on equipment as 
a means of creating a national “surge” capacity. Ideally, the types of 
equipment used by response teams across the country—protective 
equipment; chemical, biological, and radiological detectors; and 
communications equipment—would be compatible and interoperable with 
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equipment in other jurisdictions. But because standards were not available, 
we expected to find a list of recommended equipment as a resource for 
responders at all levels of government. It should be noted that in 2002, 
performance standards for civilian use of CBRN equipment were not readily 
available. PSEPC officials also pointed out that the international community 
is struggling to develop technical standards for CBRN equipment and that 
the community considers this to be a long-term problem. 


2.157 Based on its consultations with local first responders, the federal 
government knew they needed its leadership in developing a response to 
terrorism. When OCIPEP first announced that CBRN equipment would be 
funded, it said it would develop national guidelines for the acquisition of 
equipment and training—guidelines that were never developed. While 
OCIPEP had gathered some information on best practices for CBRN teams, 
it did not share the information with the provinces during the JEPP funding 
process. It offered only ad hoc advice on equipment selection. In summer 
2004, officials said they were concerned that recommending specific types of 
response equipment could create a liability for the federal government. 


2.158 Each province submitted its own proposals for equipment purchases, 
specifying the type and quantity of equipment its response teams needed. In 
the absence of federal guidelines on performance and cost, we found a wide 
variation in the types and cost of equipment purchased across the country. 


2.159 For example, the costs of Level A suits to protect against chemical and 
biological hazards ranged from $700 to $7,200, with no explanation of the 
cost difference. Different jurisdictions bought different detectors that 
provided different levels of performance at different costs. One city purchased 
a complex chemical detector for $180,000. However, its proper operation 
requires highly specialized training and ongoing maintenance and, in a recent 
incident, the municipality could not operate the detector properly and had to 
request the RCMP’s assistance. The municipality has since shelved the 
detector because it could not dedicate the resources needed to operate it 


properly. 


2.160 We found a considerable variation in the capabilities of the CBRN 
equipment purchased and in the training required for its proper operation. 
These variations would translate into problems with interoperability and 
surge capacity. We noted differences, for example, in the protection levels and 
wearing times of various suits and breathing apparatus and in the ability to 
reliably detect and quantify a broad spectrum of chemical and biological 


agents. 


2.161 A new strategy has been developed. PSEPC recently developed a 
draft national strategy for responding to CBRN incidents. Its objectives 
include working with provincial and territorial governments, standards 
organizations, and industry to develop equipment capability standards and 
guidelines for CBRN agents. To meet this objective, the CRTT and PSEPC 
will provide support for the building of a testing and evaluation capability at 
the Counter-Terrorism Technology Centre (CTTC) in Suffield, Alberta, for 
use by the first responder community, governments, and industry. 
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2.162 In October 2004, the CBRN Research and Technology Initiative 
began discussions with the CTTC the on how to conduct testing and 
evaluation of CBRN equipment and technologies. Any standards would be 
communicated by PSEPC to the first responder community. 


2.163 Recommendation. Public Safety and Emergency Preparedness Canada 
should lead the development of nationally accepted standards for equipment 
used in responding to chemical, biological, radiological, and nuclear threats. 

It should also work toward creating unified standards with the United States. 


Public Safety and Emergency Preparedness Canada’s response. Work is 
already underway, PSEPC is collaborating with the Chemical, Biological, 
Radiological and Nuclear Research and Technology Initiative (CRTI) to 
develop national chemical, biological, radiological, and nuclear (CBRN) 
equipment standards in Canada. PSEPC and the CRT] are currently 
developing a framework to guide work in this area, in co-operation with all 
key stakeholders, including other federal government departments, provinces 
and territories, the U.S. Department of Homeland Security, industry, 
standards development organizations, and first responders. 


2.164 Little training completed for first responders. Budget 2001 allocated 
$59 million over five years to the training of first responders; six departments, 
led by OCIPEP were tasked with developing a CBRN training program for 
first responders. Of the Budget funding, $11.2 million was allocated to 
OCIPEP’s Emergency Preparedness College and $21 million to Health 
Canada. We expected to find training programs based on risk assessments, 
with their delivery structured to ensure that training was timely and efficient. 


2.165 We found problems with the CBRN first-responder training program. 
Federal training has been delayed significantly, and only a small number of 
first responders have been trained in Canada’s major urban centres. The 
structure of the training is also a concern. 


2.166 The Emergency Preparedness College, in partnership with the other 
federal departments and agencies, was to design and deliver the joint CBRN 
training program. A four-level program of CBRN courses was designed, with 
the first two levels (awareness and basic training) focussed on raising 
awareness of a CBRN incident and surviving exposure to CBRN agents. The 
next two levels (intermediate and advanced) deal with intervening in and 
neutralizing the event and are directed to the traditional first responders in 
fire, police, and emergency medical services. Health care providers such as 
hospital workers were not included in the College’s specialized CBRN- 
response courses, even though the submission requesting funding for the 
training program had specifically identified the specialized training needs of 
health care workers. 


2.167 We expected that a training plan and schedule would be linked to 
threat scenarios and associated casualties, that training of higher-risk cities 
would be addressed as a priority, and that a timeline would be established for 
achieving the desired capacity. Although the College did not request a formal 
threat and risk assessment by region until April 2004, it did use available 
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intelligence to identify higher-risk cities and did give them priority in initial 
access to training. The problem that arose was in the volume of delivery of 
the training. 


2.168 The lowest levels of training are the introductory and basic courses, 
which familiarize first responders with threats but do not teach mitigation 
techniques. The joint training group estimated that about 100,000 first 
responders required the basic level of training. By October 2003, the College 
had piloted its basic course in Fredericton, New Brunswick, and in Winnipeg, 
Manitoba. The College told us it has also trained 176 trainers to give the 
basic course. To date it has issued 474 training kits to these instructors. 
Subsequently, the College planned to deliver the kits on CD-Rom. It has now 
informed us that the course will be delivered through e-learning on the Web. 
The College also proposes to deliver its introductory course on the Web. 


2.169 The higher-level courses are aimed at training first responders in how 
to work in a CBRN environment. A strategy has not been developed to link 
the training needs of a first responder team to a target level of response 
capacity. Nor has the College addressed the need for refresher training and 
retraining to allow for staff turnover. 


2.170 The course design suffered from the fact that a concept of operations 
for dealing with a CBRN event was not developed. Neither the Emergency 
Preparedness College nor its CBRN courses currently provide training in the 
incident command system, even though a number of provinces use such a 
system to govern their response to a CBRN event as well as PSEPC’s new 
National Emergency Response System (NERS). 


2.171 The funding request stated that OCIPEP would co-ordinate the CBRN 
training initiative in collaboration with the RCMP Health Canada, National 
Defence, and others. We were told that Health Canada started developing its 
own course for medical personnel in July 2003. 


2172 The request for training funds estimated the target population for the 
intermediate course at 6,000 first responders, with 2,000 targeted for the 
advanced course. The College, with its federal partners, does not have an 
adequate plan to deliver timely intermediate and advanced CBRN training to 
the first responders it identified who need training. By fall 2004, the College 
had given 134 first responders its intermediate course and 63 its advanced 
course. The College plans to provide the intermediate course in Ottawa 

five times a year and hopes to expand its offerings in the future. 


2173 Recommendation. Public Safety and Emergency Preparedness 
Canada, together with the other federal departments and agencies mandated 
to train for chemical, biological, radiological, or nuclear response, should 
revisit the objectives for providing training to first responders and the delivery 


of the training. 


Public Safety and Emergency Preparedness Canada’s response. A CBRN 
Strategic Review group has been formed and tasked with addressing key 
issues related to the development and delivery of CBRN training. The 
anticipated result is a strategic plan that will address the objectives, delivery, 
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tracking, and accountability measures for this program. The completion date 
for the plan is scheduled for the end of 2005. 


Federal strategy needed for exercising response plans 


2.174 We expected that federal response plans would be tested and exercised 
regularly and revised according to the results. 


2175 The National Security Policy has called for the staging of regular 
national and international exercises involving civilian and military resources, 
in order to assess the adequacy of the national system in various emergency 
scenarios. In our opinion, the regular testing and exercising of response plans 
is critical to their effectiveness. 


2.176 Departments have tested components of their response plans; we 
found several national exercises conducted since September 11, 2001. While 
federal departments have participated in exercises that integrated the federal 
response to a CBRN attack with the provincial and municipal responses, we 
found no federal strategy or criteria to guide the federal government’s 
participation. We found delays and gaps in the post-event analysis and 
implementing of recommendations. For example, the final analysis of the 
TOP OFF II exercise, a U.S.-Canada simulation exercise for senior officials, 
was circulated 13 months after the exercise. 


2.177 Budget 2001 did not allocate resources to exercises or to training of 
emergency managers. We could not find funds set aside subsequently at the 
program level to budget for conducting exercises. We were told that any funds 
used so far in conducting exercises were taken from other programming. For 
example, the former Department of the Solicitor General used some of its 
training money to pay for TOP OFF II exercises. 


2.178 As part of the new PSEPC structure, a National Exercise Division has 
been created. It will be responsible for developing guidelines on conducting 
exercises. 


2.179 Recommendation. Public Safety and Emergency Preparedness Canada 
should develop a long-term plan and budget for the conduct of national 
exercises. 


Public Safety and Emergency Preparedness Canada’s response. An 
Exercise Division has been created within PSEPC, and work is already 
underway to develop a long-term plan and funding strategy for a National 
Exercise Program. This will be a multi-step process that will require 
consultation and validation while continuing to deliver complex 
international initiatives such as Exercise Triple Play. A key part of the 
National Exercise Program will be the ongoing consolidation of lessons 
learned and how they inform the establishment of future priorities. 


Spending on critical infrastructure protection was not managed well 


2.180 Critical infrastructure is the backbone of Canada’s economy; it is 
essential to the health, security, safety, and economic well-being of Canadians 
and the functioning of government. Critical infrastructure includes energy 
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and utility systems, communications and information technology, financial 
systems, health care, food, water, transportation, safety services, government, 
and manufacturing capacity. 


2.181 Budget 2001 allocated $190 million to a new program for emergency 
preparedness and critical infrastructure protection (EP/CIP) co-ordinated by 
OCIPEP across 12 departments. Half of those funds were allocated to 
OCIPEP 


2.182 The purpose of the EP/CIP program was to expand the capacity of 
federal agencies to protect the country’s critical infrastructure from attack. 
We expected to see that the program funds had been allocated on the basis of 
a risk analysis. 


2.183 No threat and risk assessment was carried out before the funds were 
allocated. A committee of officials was formed that considered proposals from 
departments and agencies and made a recommendation to the Treasury 


Board. 


2.184 We looked at the three agencies that received the largest share of the 
money: OCIPER, the Communications Security Establishment, and the 
RCMP combined accounted for 80 percent ($152.5 million) of the total 
budget. We found that 


¢ the targeting of the Communications Security Establishment’s allocation 
was based on reasonable criteria; and 


* the RCMP is using its funding to expand existing cybersecurity 
programs, which is consistent with its approval from the Treasury Board. 


2.185 We found that funds allocated to OCIPEP were directed to five 
thematic initiatives: 
* putting the Government of Canada infrastructure house in order; 
* enhancing federal, provincial, and international partnerships; 
* enhancing national operational capacity; 
* developing and implementing targeted programs; and 


* strengthening the policy framework. 


What we could not find was evidence of a risk analysis to show how the 
decision was made to give OCIPEP half of the $190 million or how that 
amount was allocated to the five initiatives. 


2.186 As part of the overall program, OCIPEP was to perform a co-ordinating 
function. It was to work with the Treasury Board Secretariat to recommend 
the allocation of funds to individual departments. It was also responsible for 
reporting annually to the Treasury Board Secretariat on how the EP/CIP 


program was performing. 


2.187 At the time of our audit, OCIPEP had compiled one integrated annual 
report on the critical infrastructure protection program of the public safety 
and anti-terrorism initiative, as required. We found that since it was the 
initial report, much of the activity reported involved attendance at meetings 
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or the development of planning frameworks. OCIPEP held three meetings 
in 2003 to discuss a common evaluation framework. That framework has 
been developed. We did not find evidence that any additional co-ordinating 
meetings were held. 


2.188 Information for the second report was due from departments in 
September 2004. However, the Treasury Board Secretariat issued a revised 
reporting template on 17 December 2004. 


2.189 Officials were unable to say how much money OCIPEP had spent on 
the critical infrastructure protection program and how much funding had 
lapsed. We interviewed staff from OCIPEP National Defence, and PSEPC, 
and no one was sure whether funding had been deferred to subsequent years 
or had been absorbed into other programs. Program staff told us that in the 
first two years they thought they had lapsed $10 million of a $35 million 
budget, but we were unable to verify this. 


2.190 We were able to determine that part of the problem in tracking 
expenditures was the failure of OCIPEP staff to charge their work to the 
correct financial codes. Management could not correct this problem. In our 
opinion, basic management controls were missing. We were told that the 
program may not recover the lapsed funding. 


2.191 PSEPC has acknowledged that problems did occur in the past. It has 
undertaken to develop a co-ordinated strategic plan that establishes goals 
and priorities for enhancing capacity; to monitor the plan’s implementation; 
to identify and address gaps in emergency preparedness; and, based on 
established standards and guidelines, to evaluate the effectiveness of 
expenditures. 


2.192 We support PSEPC’s proposed solution to the problems we have 
identified. 


2.193 Recommendation. Public Safety and Emergency Preparedness Canada 
should revisit its programs for the emergency preparedness and management 
of the nation’s critical infrastructure protection and base its strategy on a risk 
assessment. 


Public Safety and Emergency Preparedness Canada’s response. PSEPC is 
currently working with its partners (other government departments, 
provinces, territories, private sector, and U.S.) to develop a national strategy 
and work plan based on risk management. A consolidated critical 
infrastructure protection (CIP) risk assessment is challenging because of the 
different states of preparedness and security of each of the 10 national critical 
infrastructure sectors. Federal officials (including PSEPC staff) have worked 
with their U.S. counterparts to develop CIP risk assessment methodologies for 
the implementation phase of the Canada-U.S. Public Security Technical 
Program (PSTP). The results of these methodologies will guide work on risk 
assessments in both countries and be one of the contributions to our 
respective and complementary national CIP strategies. 


36 


Chapter 2 


Report of the Auditor General of Canada—April 2005 


NATIONAL SECURITY IN CANADA—THE 2001 ANTI-TERRORISM INITIATIVE: 
AIR TRANSPORTATION SECURITY, MARINE SECURITY, AND EMERGENCY PREPAREDNESS 


Conclusion 


2.194 The creation of Public Safety and Emergency Preparedness Canada has 
begun the process of integrating and advancing emergency preparedness 
programs. However, due to the relative newness of the current organization, 
much remains to be done. 


2.195 Prior to the reorganization, funding for first responders under the 
Budget 2001 anti-terrorism initiative suffered from the poor sequencing of 
programs. The enhancement of response capacity is a function of procedures, 
equipment, and training, all working together. The federal training programs 
lagged behind the provinces’ equipment purchases, and the strategy to 
co-ordinate and integrate the responses of different levels of government and 
regions is still being developed. Under the initiative’s funding, each 
component was developed separately when, instead, they should have been 
developed to reinforce each other. 


2.196 Funds for first responder equipment purchases were disbursed without 
reference to a threat and risk analysis, and the result was a poor allocation of 
those funds. We found that equipment purchased for similar needs varies 
widely in cost and quality. Most critically, the opportunity was not taken to 
create a national pool of compatible, interoperable equipment. 


2.197 Although first responders cited training as their primary need, after 
significant expenditures the federal program has trained few first responders © 
at the intermediate and advanced levels. Again, threat and risk analyses were 
not used to establish program goals. 


2.198 The new Public Safety and Emergency Preparedness Canada faces 
many challenges to achieving the goals set for it by government. It will be 
important to ensure that the Department has the appropriate authorities, 
structures, and resources necessary to address current and future problems. 
Senior management is aware of both the challenges and the need to avoid the 
circumstances that contributed to the problems encountered by OCIPEP 
over the course of that organization’s existence. 


2.199 Moreover, the new department needs to review many of the decisions 
taken by OCIPEP In our opinion, without strong and clear support from all 
areas of the federal government, PSEPC will be years away from meeting the 
goals it has established and that have been established for it. And the gaps in 
Canada’s ability to respond to an emergency will remain. 


2.200 We also assessed the extensive improvements made to air transport 
security. Transport Canada did not meet our audit criteria in its oversight of 
the air transport security system. It has expanded the security inspection 
service and improved training; however, we could not conclude whether the 
number of inspectors and the frequency of inspections are appropriate, as the 
program has not been based on a documented risk analysis. The security 
inspection and enforcement regime was not designed to regulate federal 
agencies such as CATSA, the Canadian Air Transport Security Authority. 


Report of the Auditor General of Canada—April 2005 


Chapter 2 37 


NATIONAL SECURITY IN CANADA—THE 2001 ANTI-TERRORISM INITIATIVE: 
AIR TRANSPORTATION SECURITY, MARINE SECURITY, AND EMERGENCY PREPAREDNESS 


2.201. CATSA was created to manage and deliver many components of air 
travel security that were dramatically expanded by the 2001 Budget initiative. 
About $1 billion has been spent to acquire and install equipment to screen 
for and detect explosives. We found that the implementation of the 
Explosives Detection Systems program has proceeded well to date. 


2.202 We found that the marine security programs have made good progress 
and met most of our criteria. We noted that threat and risk analysis was used 
as a basis for allocating funds to priority areas. However, additional 
expenditures will likely be required to fill gaps in capabilities. The 
implementation of the International Ship and Port Facility Security Code is 
generally proceeding well, though here, too, success may depend on the 
availability of resources in the future. 
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About the Audit 


Objectives 


The objectives of the audit were to determine whether 
* Transport Canada’s oversight of the air transport security system is adequate; 


* Transport Canada and the Canadian Air Transport Security Authority (CATSA) have adequately managed 
those elements of Explosives Detection Systems acquisition and implementation for which they are responsible; 


* marine surveillance capital projects have been adequately managed and the International Ship and Port Facility 
Security Code implemented according to the government’s plan; and 


* emergency preparedness programs of the federal government are adequately managed. 


Scope and approach 


This audit was the second of two audits of the government’s National Security Enhancement Initiative presented in 
Budget 2001. The report on the first audit was tabled in March 2004. This report on the second phase addresses the 
integrity of the air transport security system as a whole; capital expenditures on Explosives Detection Systems and 
marine security surveillance systems; the implementation of the International Ship and Port Facility Security Code; 
and emergency preparedness programs. 


The audit focussed on the departments and agencies involved in implementing or monitoring these programs. They 
included 

* Canadian Air Transport Security Authority 

+ Fisheries and Oceans Canada (specifically, the Canadian Coast Guard) 

* Health Canada 

* National Defence 

* Office of Critical Infrastructure Protection and Emergency Preparedness 

* Public Health Agency of Canada 

* Public Safety and Emergency Preparedness Canada 

* RCMP 

* Solicitor General Canada 

* Treasury Board Secretariat 

¢ Transport Canada 


Our audit was limited to federal programs and did not include emergency preparedness plans and capabilities of 
other levels of government. 


Criteria 


Our audit was based on the following criteria: 
+ The air transport security system should be based on an adequate risk assessment. 
- Transport Canada should take adequate measures to ensure compliance with its security regulations, including 
— adequate training and numbers of inspectors; 
— adequate frequency of inspections; 
— quality control of inspections; 
— analysis of breaches; 


— corrective action of recurring and systemic problems. 
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+ Equipment and contracting options should have been appropriately assessed. 


« There should be time and cost controls and reporting commensurate with an undertaking whose capital costs 
exceed $100 million. 

+ Equipment acquired should meet Transport Canada’s standards and objectives as stated in acquisition plans 
and contract specifications. 

* Operators should meet Transport Canada’s standards for the detection of simulated threats and threat image 
projection. 

+ Transport Canada should mitigate the risks that implementing EDS poses for the smooth functioning of the air 
transport system as a whole. 

* Projects should comply with best practices and standards of project management. 


* Departments should ensure that best practices for contracting are observed and that contracting regulations 
are followed. 


* There should be a clear chain of command for the federal and national response to incidents. 
* Federal response plans should be based on threat/risk assessments. 
* Federal response capabilities should be matched to approved threat and casualty scenarios. 


* There should be federal equipment guidelines for first responders, specifying minimum performance and 
interoperability requirements as well as test protocols. 


- First responder training should 


be linked to current threat/risk assessments; 


be timely; 


be targeted and prioritized; 


| 


include cyclical retraining. 
* Federal response plans should 
— be tested and evaluated regularly; 
— be integrated across levels of government; 


- be revised according to post-event and post-exercise analysis. 


Audit team 


Assistant Auditor General: Hugh McRoberts 
Senior Principal: Peter Kasurak 

Principal: Gordon Stock 

Director: Edward Wood 


Dawn-Alee Fowler 
Anthony Levita 
Carol McCalla 
Mark Carroll 

Sami Hannoush 
Steven Mariani 
Jacinthe Pépin 


For information, please contact Communications at (613) 995-3708 or 1-888-761-5953 (toll-free). 
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Passport Office 


Passport Services 


All of the audit work in this chapter was conducted in accordance with the standards for assurance engagements set by the 
Canadian Institute of Chartered Accountants. While the Office adopts these standards as the minimum requirement for our audits, 
we also draw upon the standards and practices of other disciplines. 
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Foreign Affairs Canada and the Passport Office have provided leadership on addressing 
identity documentation issues 


Challenges to the Security, Policy and Entitlement Division 


Control over issuance of passports by missions abroad presents several risks 


Service to the public 


Key service standards exists but gaps remain 

Costs have risen significantly 

There Is little contingency planning for crises 

Inability to forecast and influence demand places an undue burden on service 


Management of improvement initiatives 
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Main Points 


31 The Passport Office is struggling to meet increasing security 
expectations and demands for service. Significant improvements are 
necessary in the processes for determining passport entitlement. The Office is 
currently unable to fulfill all of its responsibilities under the Canadian 
Passport Order. Its watch list is deficient and often not updated in a timely 
fashion because the Office has not found ways to automatically obtain data 
from other government sources. Management does not sufficiently monitor 
some key security functions to ensure that they are properly carried out. 


3.2 The Passport Office has established and achieved its published service 
standards for clients. However, costs are increasing and are not tied to service 
standards. Although it has reported on the performance of certain major 
service standards, there is limited performance information about security 
measures. At the time of the audit, the Passport Office did not have the 
management systems and practices that would prepare it to meet future 
challenges effectively. 


3.3. The Passport Office and Foreign Affairs Canada did not meet the 
requirements for ongoing consultation on service standards and how they 
relate to passport and consular fees. The Passport Office also lacks reliable 
cost information related to published service standards. 


Background and other observations 


34 The Passport Office is a Special Operating Agency reporting to Foreign 
Affairs Canada. It finances its operations from the user fees it collects. The 
Office issued about 2.5 million passports in 2003-04. It collected revenues of 
$158 million that it used to finance its operations and had a net income of 
$9 million. It also collected $54 million in consular fees for deposit to the 
Consolidated Revenue Fund. The Passport Office is responsible for enforcing 
the Canadian Passport Order in issuing various travel documents. Issuing the 
24-page blue passport, which in 2005 rose in cost from $85 to $87, makes up 
about 98 percent of Passport Office business. 


The Passport Office has responded. In its response to each recommendation 
throughout the chapter, the Passport Office indicates the action it has taken, 


is taking, or plans to take to address the recommendation. 
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Special operating agency—An agency within a 
government department that has greater 
management flexibility in return for certain levels 
of performance and results. 


Revolving fund—A fund provided for a 
particular purpose that Is periodically 
replenished by transfers from other funds or, as 
in this case, by user fees. 
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Introduction 


35 The Canadian economy depends on trade. But globalization—of 
production, commerce, crime, terrorism, and travel—is evolving rapidly and 
creating new opportunities, risks, and requirements for travellers. Still, 
Canadians need and expect an official travel document that is secure and 
universally respected to ease their way across borders and around the globe. 
The Canadian Passport Order defines a passport as an official Canadian 
document that shows the identity and nationality of a person for the purpose 
of facilitating travel by that person outside Canada. 


3.6 Rising demand and the security challenges due to the events of 
September 11, 2001, also contribute to an environment where the pace, 
scope, and direction of change is increasingly unpredictable. The Passport 
Office must balance three essential factors—cost, service, and security— 
while investing in the technology and human resources required to meet the 
needs of coming years. 


3.7 The Passport Office is responsible for issuing, revoking, and 
withholding or recovering Canadian passports issued to Canadian citizens 
and other travel documents to Canadian permanent residents. Issuing the 
24-page blue passport makes up over 98 percent of its business. 


38 The Passport Office is a special operating agency reporting to Foreign 
Affairs Canada and, as such, is expected to 


* encourage innovation in administration and service delivery while 
respecting control requirements and managing risk, 


* provide greater freedom from departmental and government-wide 
administrative rules in return for clear performance levels, and 


* improve cost-effectiveness and service delivery by applying private 
sector techniques while tightly managing performance targets. 


39 The Passport Office operates on a revolving fund basis and is self- 
financing through user fees charged for various travel documents. Unlike 
government departments, it has no annual parliamentary appropriation for 
operations and must generate sufficient revenues to meet its expenses. It has 
a significant degree of autonomy in administering its operations. 


The Passport Office faces new demands 


3.10 The Passport Office experienced steady growth in the mid-1990s. The 
tragic events of September 11, 2001, brought security matters to the fore. 
The Passport Office now faces some serious challenges in a period of 
increased demand for passports: rising costs of production; pressures to adopt 
new technologies that heighten security features of the travel document; and 


identity authentication issues. 


3.11 In May 2004, Foreign Affairs appointed a new chief executive officer. In 
the fall of 2004, the Passport Office was considering a new strategic direction 
that would focus on two types of clients: those who are applying for their first 
passport, and those who have previously sought and obtained a passport. 
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3.12 Since September 11, 2001, Canadians are more likely to carry their 
passports when traveling to the U.S. That, along with the implementation of 
the “one person, one passport policy,” where children can no longer be 
included on a parent’s passport, have given rise to higher demand for 
passports. At the same time, there has been a worldwide move to greater 
tightening of security. 


* In 2002-03, 2.2 million passports were issued, a 13 percent increase 
over the previous year. 


* In 2003-04 over 2.5 million passports were issued, a 15 percent increase 
over the previous year. 


- The Passport Office estimates that 2.8 million passports will be issued 


in 2004-05, up from 1.0 million in as recently as 1999-2000. 


3.13 Points of service. In 2004, more points of service were opened in 
response to increased volumes. Most Canadians obtain their passports by 
applying at one of the 30 domestic offices that offer front-counter service to 
applicants. The offices are administered through four regional directorates. In 
the last few years, the Passport Office has entered into arrangements with 
Canada Post Corporation to help passport applicants: postal staff in 

49 selected locations review applications and forward complete ones to the 
Office. Canada Post charges an additional $15 fee for this optional service. 


3.14 The public can also mail completed applications to Passport 
headquarters directly. The Passport Office also accepts applications through 
members of Parliament who forwarded about 65,700 applications to the 
Office in 2003-04. In addition to its network of offices, the Office is using 
other receiving agents such as the staff of Human Resources and Skills 
Development Canada. Foreign Affairs Canada consular staff provide passport 
services abroad at over 100 missions. 


Focus of the audit 


3.15 The objectives of this performance audit were to determine whether 
the Passport Office has designed, implemented, and monitored effective 
controls over the issuance of passports; whether it has established and 
achieved reasonable levels of service; and whether it has achieved both 
objectives at a reasonable cost. 


3.16 We examined four areas in this audit: 

* security and identity verification, 

* service to the public, 

* management of improvement initiatives, and 

- fee increases. 
3.17 We undertook fieldwork at headquarters and in 15 offices. We 
examined relevant business, information technology, and management 


processes and practices, and we assessed compliance with applicable rules and 
authorities. 
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3.18 More information on the audit is found in About the Audit at the end 
of the chapter. 


Observations and Recommendations 


3.19 The production process and examiner’s role. The Canadian Passport 
Order, approved by the Governor-in-Council, specifies who is entitled to a 
Canadian Passport and the conditions that must be met. The Passport Office 
may refuse to grant a passport to Canadian citizens with certain Criminal 
Code violations or charges pending. Passport Office staff are responsible for 
applying the rules and procedures of entitlement, which are listed in the 
Canadian Passport Order (Exhibit 3.1). Applicants must be Canadian citizens 
and applicants for passports for children must have guardianship rights. 


Exhibit 3.1 Excerpt from the Canadian Passport Order 


Under section 9 of the Canadian Passport Order, the Passport Office may refuse to 
issue a passport to an applicant who: 


a) fails to provide the Passport Office with a duly completed application for a passport 
or with the information and material that is required or requested ... 


b) stands charged in Canada with the commission of an indictable offence; 
c) stands charged outside Canada with the commission of any offence that would, Ls 
committed in Canada, constitute an indictable offence; 
d) is serving a term of imprisonment or is forbidden to leave Canada by 


i) the terms and conditions of any parole or mandatory supervision imposed 
under or by virtue of Part II of the Corrections and Conditional Release Act, 


ii) the conditions of a probation order made under the Criminal Code, or 
iii) the conditions of the grant of temporary absence without escort from a prison 
or penitentiary; 


e) has been convicted of an offence under section 57 of the Criminal Code; 

f) is indebted to the Crown for expenses related to repatriation to Canada or for other 
consular financial assistance provided abroad at his request by the Government of 
Canada; or 

g) has been issued a passport that has not expired and has not been revoked. 


Under section 10 of the Canadian Passport Order, the Passport Office may revoke a 
passport for any of the abovementioned grounds and to an applicant who: 


being outside Canada, stands charged in a foreign country or state with the 
commission of any offence that would constitute an indictable offence if committed 


in Canada; 

uses the passport to assist him in committing an indictable offence in Canada or 
any offence in a foreign country or state that would constitute an indictable offence 
if committed in Canada; 

permits another person to use the passport; 

has obtained the passport by means of false or misleading information; or 

has ceased to be a Canadian citizen. 

Finally, the Minister of Foreign Affairs may refuse or revoke a passport if the Minister is 
of the opinion that such action is necessary for the national security of Canada or 
another country. 
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3.20. The granting of a passport goes through three phases: application data 
entry and authentication of the identity and citizenship of the applicant; 
certain security checks that could prevent the granting of the passport; and 
printing and mailing (Exhibit 3.2). Examiners take, on average, 8 to 

12 minutes to process a routine application. 


3.21. The Canadian Passport is valid for five years. While many other 
western nations use a ten-year period, some of those countries are considering 
a five-year validity period. A shorter validity period allows the Passport Office 
to more rapidly introduce technological enhancements that improve the 
security of the passport document, thus reducing the risk of counterfeit or 
fraudulent passports in circulation. However, a shorter validity period has 
significant cost and service implications for its operations. 


3.22. The passport application package includes information that is intended 
to help examiners assess that applicants are not only Canadian citizens, but 
that they are also the individuals they purport to be. In addition to proof of 
citizenship by either birth or naturalization, applicants must include other 
proofs of identity, guarantor information (including a guarantor-signed 
photograph), and two references. The original proof of citizenship, birth 
certificate, and proof of identity, in most cases, are compared to the 
application information and returned immediately to the applicant at the 
counter (or with the passport, for mailed applications). 


Examiners are well trained 


3.23 Examiners are the key officers in the passport entitlement process. 
Their role is to examine each application and assess whether applicants are 
who they say they are and, if so, whether they are eligible for a passport under 
the Canadian Passport Order. A national standard of performance for quality 
and efficiency governing this process only exists for examiners in training. 


3.24 We found that the initial training provided to examiners in Canada is 
comprehensive. Training for domestic examiners consists of 4 weeks of 
in-class instruction followed by a 12-week period of qualification working 
with a mentor. (Training of consular officers abroad is discussed in 
paragraph 3.57). 


3.25 We sent an interview questionnaire to 50 examiners and interviewed 
some additional examiners in the field. Most respondents expressed overall 
satisfaction with the initial training. The examiners felt that more frequent 
refresher training is required to keep up with changes in policies and 
procedures. Examiners also appreciated training on how to detect fraudulent 
documents when this was made available. 


3.26 The extensive training program requires considerable investment. New 
staff must complete training before they can handle many of their duties. 
Management plans to change the structure of the initial training to increase 
the focus on security matters. 
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Exhibit 3.2 Process and controls for issuing a passport in 2004 


Presenting the application 


Individuals take their applications to 
passport offices (walk-in or mail) or 
receiving agents, which include 
parliamentarians’ offices and certain 
postal outlets and Human Resources 
and Skills Development Canada 
offices. There, 


¢ contents noted for completeness, 
and 


* payment processed. 


Security check 


* all system alerts automatically 
processed 


Data entry 


¢ basic information entered 
electronically, 


* application digitally scanned, and * all referrals decided here 


¢ documents returned to applicant * suspect cases are tracked 


* liaison with other law 
enforcement agencies 


Problems 
OF suspicions 


Entitlement examination 


Some examples of work done by ; : 
examiners: Data integrity check 
Ensures that information that will 
appear in passport agrees with 
application and that photo and 
signature appear clearly. Any 
problems can be rectified only by 


the examiners. 


Printing 
* blank book inventory subject 
to controls 


evaluate authenticity of proof of 
citizenship document; 


check guarantor by phone; h@omnene 


ensure old passport is included 
with the application; 


check that all documents on 
the checklist are in application 
package to be forwarded onward; 


query system watch lists for any 
alerts; 


* refer any problems or suspicions 


to Security division; and «printing and malin 


(subject to Quality Assurance). 


* approve entitlement. 


CANADA 


Note: For missions abroad, 
there are about 100,000 
passports issued annually; 
process, security checks, 
and controls differ from 
applications made in Canada. PASSPORT 


PASSEPORT 
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Identity authentication and 
entitlement decisions 


Security clearance levels need to be reviewed and updated as required 


3.27. The governing authority for security clearance is the Treasury Board of 
Canada’s Government Security Policy and the Personnel Security Standard. 
y y y 
Examiners must clear the appropriate security level to be hired. The Passport 
Office determines the security level required for each examiner according to 
y | g 
the type of assets or information that the examiner will need to access. In 
general, most examiners are cleared for “enhanced reliability.” Each level 
higher requires a more thorough background check of the employee. 
s | g g ploy 


3.28. We reviewed employee personnel records to determine whether 
employees had the appropriate levels of security clearance according to the 
Government Security Policy. We also visited local offices and observed 
employee access to critical materials and equipment such as blank passports 
and printers. We found that 565 indeterminate and term examiners in the 
human resources system were cleared for enhanced reliability and 179 had 
the next higher clearance, “secret.” Of these 565 examiners, 45 had “full job 
concept” in their job description, which means they had access to critical 
material and equipment. We also observed in our visits that certain examiners 
without a secret clearance had access to critical assets in some offices. We are 
concerned that contrary to government policy, individuals with inadequate 
security clearance potentially had access to some critical assets. The Passport 
Office has informed us that this identified vulnerability has been addressed. 


Quality assurance of the examiner function is lacking 


3.29. The quality of security features on identity documents accepted by the 
Passport Office to determine entitlement varies greatly. Some older birth and 
citizenship certificates have few, if any, security features; the latest citizenship 
card, which includes several embedded security features, is more difficult to 
alter. 


3.30. The Passport Office lacks a formal quality assurance system for the 
examiner entitlement functions. Therefore, there is no assurance that 
examiners have followed the required procedures to determine that an 
applicant is eligible for a passport. In our view, the Passport Office needs a 
formal quality assurance system that checks whether an examiner is making 
the proper decisions about the identity documents presented by an applicant. 
That system should also provide an independent check over other 
entitlement functions such as verifying information provided by guarantors. 
A proper quality assurance system would ensure that any examiner making a 
systematic error in judgment would be rapidly identified and corrective action 
or training undertaken. 


Examiners do not have all the proper tools readily available 


3.31. During our visits to passport offices, we observed the type and 
availability of some key tools for examiners. Examiners at all offices had 
reference manuals and these were reasonably accessible. However, we found 
the following: 
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* A black light is needed to see the security features on the latest 
Canadian citizenship card. We observed one or two black lights 
available in some of the offices we visited, but they were not in the 
examiner area. This means that examiners have to leave their stations to 
verify the authenticity of modern citizenship cards. Examiners informed 
us that service requirements kept them from using this tool routinely, 
because doing so would cause delays. As a result, they rarely use them. 


* Magnifying glasses for document examination were generally not part of 
examiners’ standard equipment. According to their training on 
document fraud, this tool is useful in detecting some forms of document 


fraud. 


The Passport Office informed us that it will purchase black lights and 
magnifying glasses for all examiners to keep at their workstations. The Office 
also informed us that managers have been advised that they will be 
responsible for providing refresher training to examiners once the equipment 
has been delivered, and that training guides will be completed for distribution 


by 31 March 2005. 


Checks on guarantor information are not performed as required 


3.32. Guarantors are either members of certain professional associations or 
individuals with certain accreditations who confirm the identity of the 
applicant by signing a declaration and one photograph of the applicant. Each 
guarantor must have known the individual for at least two years. Applicants 
who do not know an eligible guarantor must have their applications 
notarized. The signature of a guarantor is one of the main controls to ensure 
the authenticity of an applicant’s identity. 


3.33. The examiner’s training manual includes a comprehensive list of 
criteria to validate the guarantor’s credentials or the information provided; 
this procedure is called a guarantor check. The list provides instructions on 
what should be documented about the verification performed. Generally, to 
perform a guarantor check, the examiner verifies the guarantor’s status and 
contacts the guarantor by phone to confirm the information provided. 


3.34 Passport Office procedures require that examiners conduct guarantor 
checks on a significant percentage of applications. However, there is no 
national reporting on this activity, and the Passport Office could not 
demonstrate that this objective was achieved. We examined a sample of 

50 applications where a guarantor check was mandatory, for compliance with 
the requirement. In 37 cases, we found no evidence of a guarantor check. 
Part of the reason for this non-compliance appeared to be poor 
communication of corporate requirements on one of the categories. We then 
examined a second sample of 50 applications for another category where the 
requirements were clearly communicated. We found no evidence of a 
guarantor check in 2 of those cases. For both categories, the results of the 
checks performed were poorly documented. As a result of our audit, the 
Passport Office has informed us that it has sent policy clarification to its 
regional offices and will ensure that the policy is adhered to. 
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Confirming vital statistics is difficult for examiners 


3.35 At their discretion, examiners confirm the data on a birth or citizenship 
certificate by asking regional co-ordinators to contact Citizenship and 
Immigration Canada or the relevant provincial registry and seek confirmation 
that the presented identity document was issued with identical data. This is 
time consuming and non-routine. The Passport Office conducted a pilot 
project in one office where the data were automatically checked against the 
provincial registry through an electronic link. The pilot proved successful, 
and participants told us that this type of electronic link is ideal to ensure the 
validity of birth or citizenship certificate information. 


3.36 Currently, there are no direct electronic links to provincial bureaus of 
vital statistics. Such links would provide immediate confirmation of data and 
allow examiners to check for reported deaths. No similar link exists between 
the Passport Office and Citizenship and Immigration Canada. Such a link 
would even permit photo verification of some naturalized citizens. Together 
with other federal and provincial/territorial partners, the Passport Office is 
looking into the development of a national routing system to create such 
electronic links. 


Foreign Affairs Canada and the Passport Office have provided leadership on addressing 
identity documentation issues 


3.37 The Passport Office and Foreign Affairs Canada have provided 
leadership on addressing identity documentation issues in Canada. As a result 
of such issues and growing security concerns after the events of 

September 11, 2001, the federal government spearheaded the formation of 
the Council of Identity. Its objectives were to develop a policy framework on 
identity in Canada, facilitate implementation of the policy, and monitor its 
effectiveness. The Council is chaired by the assistant deputy minister of 
Foreign Affairs and is composed of representatives from various federal 
agencies and provincial and territorial vital statistics personnel. The Passport 
Office is a leading participant in the Council as part of its efforts to improve 
its ability to rely on foundation documents (for example, birth, death, 
marriage, and citizenship certificates) presented by applicants, regardless of 
the level of government in Canada that produces them. 


3.38 = Since April 2002, the Council has held six meetings. By the spring 

of 2004 it had met its main objective: the completed development of a 
framework for a comprehensive and consistent approach to establishing and 
verifying identity across federal and provincial/territorial jurisdictions. 

It submitted the framework to the government for approval. The framework 
provides the policy parameters for authentication, secure registration, and 
verification of identities in Canada, based on the concept of foundation 
documents. At the time of our audit, Council members were awaiting the 
framework’s approval. Such a framework is considered necessary for the 
creation of a national routing system, which would allow electronic 
authentication of foundation documents. 
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3.39 The Security, Policy and Entitlement Division of the Passport Office 
has 62 staff and is responsible for all security matters related to personnel and 
the production and physical security elements of passports. Within the 
division, the Security Operations unit is responsible for the functions that 
assist in making and supporting entitlement and passport issuance decisions. 


3.40 In order to assess the adequacy of the Security Operations’ activities, 
we asked the Passport Office to provide us with management reports. We also 
sought to identify any systematic quality assurance controls over security 
activities. The Passport Office advised us that they do not yet have 
aggregated management reports on all the activities of the unit. They have 
problems with extracting management information from the main 
information systems. Management advised us that supervisors could assess 
some activities. However, the tracked data is not integrated and analyzed in a 
way that would assist them in monitoring the performance of the division, in 
making resource allocation decisions, or in making decisions on program 
priorities. 


3.41 Access to the passport issuing system needs to be restricted. We 
found very poor control over the granting of access rights to users. Data 
administrators, system administrators, case management officers, entitlement 
review analysts, and operational and non-operational office managers all have 
user access profiles that allow them to issue a passport. We also found 
duplicate, system, or generic user profiles. Of 71 unsuccessful examiner 
candidates sampled, 19 still had active user access rights that would allow 
them to approve an application. 


3.42 Incases where there is no automatic alert in the system, examiners can 
authorize the printing of a passport without further checks on their decision. 
We expected that access to the passport production and information system 
would be adequately safeguarded by assigning the appropriate profile to each 
user name. We expected that only qualified examiners currently dedicated to 
examiner functions would be able to access the system to produce passports. 
We found that there were a significant number of users who are not 
examiners and who have access to the system. They can approve entitlement 
and, in some cases, effectively trigger the issuance of a passport. This puts the 
issuance process at risk. The Passport Office informed us that this is, in part, a 
result of new workflows developed for high volume offices where data entry 
have access to printing due to the system 


staff—who are not examiners 
design. 


3.43 The Passport Office cannot effectively enforce certain provisions of 
sections 9 and 10 of the Canadian Passport Order. Prior to approving the 
issuance of a passport, examiners automatically check all applicants against 
the watch list from the Passport Office’s information system to detect 
potential reasons for refusing a passport. This automatic check is the main 
security control in the process; there is no information related to criminal 
charges or probation orders required on the application form. If applicants do 
not have alerts against their names on the automated watch list, examiners 


have no way of knowing whether decisions need to be made under section 9. 
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Similarly, security personnel cannot know if they should consider revoking a 
person’s passport under section 10. In addition, we could not find any 
documented criteria for making discretionary decisions to refuse or revoke a 
passport. 


3.44 With its current information, the Passport Office cannot properly 
enforce certain provisions of sections 9 and 10 of the Canadian Passport 
Order, which address individuals with criminal charges or convictions 
(Exhibit 3.1). The Passport Office has three components to the watch list in 
its system: lookouts for persons, alerts for documentary evidence of 
citizenship, and alerts for guarantors. As of August 2004, the persons watch 
list had about 73,000 entries while the documentary evidence of citizenship 
and guarantor alerts had about 2,800 and 290 entries respectively. The largest 
category within the watch list is for parents (22,800) who are in arrears for 
child support payments. 


3.45 Security officers advised us that the list is developed from information 
provided by a variety of sources, including migration integrity officers from 
missions abroad, law enforcement officers, courts, airline staff, and examiners. 
The majority of the information is provided voluntarily, on an informal basis. 
There are no automated data-sharing processes. 


3.46 There are several databases maintained by government departments 
and agencies that the Passport Office could try and access to make its watch 
list more complete. For example, the Canadian Police Information Centre 
(CPIC) lists persons wanted or on probation. We are not suggesting that the 
Passport Office must refuse or revoke passports for all of these people; many 
are likely not of interest to the Office in terms of enforcing the Canadian 
Passport Order. However, the Passport Office’s watch list includes fewer than 
2,000 of those individuals who are wanted or are on probation. The current 
watch list appears to be of limited value in enforcing sections 9 and 10 of the 
Passport Order. The Office has not found a way to gain automated access to 
these important data. The Passport Office staff may manually access the 
CPIC database through a terminal at its headquarters to examine any 
individual application. However, it does not electronically link the CPIC data 
to its own watch list. Since its system lacks an electronic link to this key data, 
the automatic checks would not flag pertinent applications for review. We 
have been informed that other agencies have established such links for their 
own specific needs. We recognize that the Passport Office needs the 
co-operation of other government departments to obtain much of the data it 
needs. Nevertheless, without reasonable automated links to other 
government data systems that deal with criminal offences, its watch list will 
likely remain deficient. 


3.47. We also found several weaknesses in the management of the watch list. 
The updating of the list was neither complete nor timely, in part due to 
problems co-ordinating with other government departments and agencies. 
For example, in 45 investigations we examined that were performed by the 
Security Operations unit, an average of 18 days elapsed between notification 
and creation of alerts. 
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3.48 Under section 10 of the Passport Order, the Passport Office may revoke 
the passport of anyone charged with an indictable offence, sentenced to jail, 

or released on parole. The Passport Office has revoked less than 25 passports, 
on average, each year for the last three years. 


3.49 For examiners and security staff, the automated watch list is a key 
control. When the watch list is deficient, as is currently the case, they cannot 
properly fulfil their responsibilities and enforce important sections of the 
Canadian Passport Order that deal with persons who are charged with crimes, 
serving sentences, or are on parole. 


3.50 | Weaknesses in the investigations of sensitive cases. The Security 
Operations unit collects information and conducts investigations on any 
suspicious activity. Files are created when information such as the following is 
received: . 


* a Canadian has been identified as a smuggler or has committed passport 
fraud and there is sufficient evidence to pursue an investigation, 


* an individual obtained a passport fraudulently, 


* an applicant forged guarantor information and charges will be pursued, 
or 


* an applicant forged the other parent’s signature and charges will be 
pursued. 


3.51 In the absence of an effective case management system, the unit has ~ 
only some manual records of its investigative activity. Such records do not 
allow the Passport Office to assess the outstanding workload, the rate of 
effective resolutions of investigations, or resource requirements. 


3.52 We examined a sample of 50 sensitive case files and observed several 
problems. For example, 


* in our view, the Passport Office did not fully investigate 12 cases. 
According to the Office, 3 of the 12 investigations were incomplete due 
to resource limitations, and 6 resulted in the Office granting or 
extending a limited validity period. 


* in our view, the Office should have referred at least 6 cases to the Royal 


Canadian Mounted Police (RCMP). 


© for 10 cases that the Office did refer to the RCMP there was no 
documentation on any follow-up activity to determine the status or the 
outcome of the RCMP’s investigations. 


3.53 Follow-up with the RCMP to determine the outcome of their 
investigations could provide useful lessons learned for the Passport Office in 
disposing of other outstanding cases. It would be in a better position to refer 
only those cases worthy of further investigation by the RCMP. The Passport 
Office and the RCMP could then better utilize their respective resources. 


3.54 The Passport Office has improved its recording and sharing of 
information on lost and stolen passports. We followed up on the progress 
made as a result of observations on lost and stolen passports in our 
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March 2004 Report, National Security in Canada—The 2001 Anti-Terrorism 
Initiative. The Passport Office has made significant progress. 


3.55 In February 2004, the Passport Office assumed responsibility for 
entering its data into the Canadian Police Information Centre database. For 
those files we examined, it still took an average of 35 days after a passport was 
reported lost or stolen (down from 70 days noted in our March 2004 Report) 
to give that information to CPIC. The Passport Office has recently automated 
several procedures that should eliminate the delays. Furthermore, there is 
currently no backlog of files in the Passport Office for entry to the CPIC 
database, compared to over 4,000 observed in our March 2004 Report. 


3.56 ‘To be effective, information on lost and stolen passports needs to be 
recorded and shared in a timely fashion. As a result of our March 2004 audit, 
the Passport Office is now sharing information with the Canada Border 
Services Agency. If a border services officer scans or queries a passport that 
the Office has recorded as lost or stolen, it will trigger an alert. The Passport 
Office has also begun to share the information with the International 


Criminal Police Organization (INTERPOL). 


Control over issuance of passports by missions abroad presents several risks 


3.57 The Passport Office relies on Foreign Affairs Canada consular staff to 
issue over 100,000 passports each year to applicants abroad. Consular staff 
perform examination, approval, and printing procedures. There are several 
key differences between mission and domestic passport services that result in 
increased risk: 


¢ Consular staff do not consistently receive the same formal training and 
qualification as domestic examiners. Their passport training typically 
amounts to two days, compared with the four weeks of classroom 
training and twelve-week mentoring period provided domestically. 


* In 2004, the upload of mission information into the Passport Office’s 
system began on a daily basis. Previously, this was done every four to 
six weeks. 


* Missions still print passports using the older passport booklet, which 
does not have the higher security features offered by the new booklet 
and new printing technology. The Passport Office is now planning to 
repatriate the printing of booklets, while providing only emergency 
travel documents at missions. 


3.58 Recommendation. Foreign Affairs Canada and the Passport Office 
should do the following: 


* Conduct a detailed security risk assessment of the passport examination 
and security functions, identify important gaps in the fulfilment of its 
mandate under the Passport Order, and take appropriate action. Such an 
assessment should address Passport Office examiners’ security 
clearances, the conduct and recording of guarantor checks, system 
profiles, the completeness and accuracy of its watch list and methods 


14 Chapter 3 Report of the Auditor General of Canada—April 2005 


PASSPORT OFFICE—PASSPORT SERVICES 


used to share data with other federal and provincial agencies, its lessons 
learned process, and its management of investigations. 


* Introduce quality assurance and control measures over its key 
entitlement operations to ensure that the Office conducts proper 
internal checks on initial decisions and properly controls access to 
critical assets. 


* Work toward establishing electronic links with Citizenship and 
Immigration Canada and the provinces that could automatically 
validate the identity data on an application. 


+ Provide examiners with the proper tools to discharge their 
responsibilities. 


* Provide more thorough examiner training for consular officers. 


¢ Work with Citizenship and Immigration Canada, the RCMP and the 
Canadian Border Services Agency to develop memoranda of 
understanding for the sharing of information vital to the enforcement of 
the Canadian Passport Order. 


Passport Office’s response. As a result of increased international and 

domestic attention to security, there has been a significant shift in the policy 
framework in which the Passport Office operates. In the past, this framework 
primarily emphasized the delivery of service. Now it is rooted in domestic and 
global security, guided by the Canada National Security Policy of April 2004. 


The Passport Office has put in place a series of key initiatives to address this 
recommendation. Specifically, working with Foreign Affairs Canada, it has 
launched a risk assessment to address strategic risk, including in the area 

of security. Once these results are received, a more focussed security risk 
assessment will be conducted. This will be accomplished in fiscal year 


2005-06. 


In fiscal year 2004-05, the Passport Office initiated a review of its Security 
Bureau and its functions to further enhance passport integrity, and it has 
allocated additional resources in support of this critical function. In fiscal year 
2004-05 the budget of the Security Bureau was augmented by 25 percent 
over fiscal year 2003-04. The Bureau has added the positions of manager of 
intelligence and manager of compliance. 


The Passport Office is redefining the mandate of the Security Bureau to 
identify core functions and processes. From this review, a three-year strategic 
plan and restructuring recommendation will be developed to narrow the 
focus of several divisions while introducing experienced senior managers and 
professionalizing staff development and training. The plan will be in place in 


fiscal year 2005-06. 


The creation of new Regional Security units reporting to both regional and 
Passport Office headquarters will further ensure that the security continuum, 
from identity verification to passport printing, is strengthened. In fiscal year 
2005-06, eight new regional security officers will be hired for this purpose. 
The units will be located in regional headquarters across the country and will 
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Service to the public 


be used to support the compliance program, conduct on-site interviews of 
applicants with complex cases, and investigate fraudulent applicants or 
applications. 


As of 4 February 2005, the Passport Office has purged all inactive and 
duplicate access to its system. A profile compliance monitoring system of each 
non-examiner’s access is currently under development. 


An expanded trial to establish electronic links for verification of identity data, 
involving several federal government departments and two provinces, is 
scheduled for launch in April 2005. This system will be further developed as 
funds become available. 


The Passport Office is researching case management software with the intent 
of having a security and intelligence case management system in place by 
September 2005. This important tool, along with facial recognition 
technology, will enable management to strengthen the process of verifying 
identity before the issuance of a Canadian passport. 


Nevertheless, the fundamental challenge faced by the Passport Office will 
always lie in correctly confirming an individual’s identity. Canada does not 
have an explicit identity program. None of the documents commonly 
accepted in Canada as proof of identity was originally intended for that 
purpose. A second challenge is the ability to share information among federal 
departments and agencies, because of issues of technical interoperability and 
legislative frameworks. The Passport Office and Correctional Service Canada 
are now close to concluding a Memorandum of Understanding for the sharing 
of critical information. 


3.59 The Treasury Board’s Results for Canadians: A Management Framework 
for the Government of Canada sets out the key principles of public services 
delivery. The framework states: 


As citizens, Canadians have a right to fair, equitable and 
reasonable treatment from federal government institutions. As 
clients, Canadians have a right to accessible service that meets 
their priorities for improvement. .. . 


Citizens want the government to respond to their needs and 
provide choice: one-stop, integrated access via Internet, 
telephone or mail, or in person. 


3.60 The Treasury Board’s policy on external charging required the Passport 
Office, in delivering its service, to conduct open, transparent, and 
informative consultations with stakeholders. It must consult its stakeholders 
on costing, service delivery methods, and on establishing standards. 


Key service standards exists but gaps remain 


3.61 The Passport Office manages its operations using some meaningful key 
standards, including public wait times for walk-in service, phone calls, and 
turnaround times for receiving a passport. These standards serve as good 
indicators of performance and as useful tools for service improvement by the 
agency. 
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3.62 However, the service standards were lowered in December 2001 
without the required consultation with stakeholders. For instance, 
turnaround target times increased in 2001 from 5 to 10 days for 
over-the-counter applications and from 10 to 20 days for mailed applications. 


3.63 = Standards for such examiner functions as identity verification and 
citizenship determination do not exist. There is also no national standard on 
examiner output and error rate—the number of applications examiners are 
expected to process each day and the number of errors that could be 
expected. 


3.64 The Passport Office has reported on its performance in some relevant 
and meaningful ways: 


* using some key performance standards (such as wait times and 
turnaround times); 


* discussing and explaining how expected performance was being met or 
was not; and 


* including negative information in their reporting. 


3.65 When the Passport Office was established as a Special Operating 
Agency in 1990, the Treasury Board approved a document outlining its 
performance accountability. The business plans of the Passport Office further 
elaborate on three major areas of performance: service to the public, security, 
and stability of its operations. It has included extensive information in its 
annual reports about its service to the public. However, there is limited 
performance information to Parliament about security and the stability of its 
operations. 


3.66 The Passport Office follows the good practice of publishing some client 
satisfaction survey results. Improvements could be made to the presentation 
of survey information; for example, it could comment on any data limitations 
so that readers can interpret the survey results more accurately. Nevertheless, 
performance information found in the Office’s Annual Report does not 
adequately reflect the three focuses of its business plan: reduce passport fraud 
(security); provide reliable and convenient service; and put in place the 
conditions necessary for improvement (stability). Reporting on security and 
stability activities is missing or insufficient. We also found that it does not 
report consistently on phone service standards and performance. 


Costs have risen significantly 


3.67 The number of passports processed by each employee has been 
dropping consistently over the past five years. A decrease in productivity can 
have a direct impact on service to clients. Lower outputs by employees can 
mean longer waiting times for the applicants. 


3.68 The Passport Office tracks the productivity of its offices, which varies 
significantly. Generally speaking, productivity in medium offices is similar, but 
productivity varies greatly among small and large offices. The Passport Office 
has not identified the reasons for those variations. Doing so would permit it to 
adopt corrective measures to control costs. At the time of our audit, the 
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Passport Office was developing a cost information system to better control its 
costs. The Treasury Board policy on fees charged to the public requires that 
departments and agencies consult clients on establishing service standards 
and the cost of those services. Currently, the Passport Office does not have 
adequate cost information on meeting service standards and, consequently, it 
cannot consult stakeholders about costs of regular or expedited services. 


There is little contingency planning for crises 


3.69 The Passport Office has experienced significant volume increases over 
the past few years. The increases of 2002—03 and 2003—04 were unusual, and 
the Greater Toronto Area experienced unprecedented demand. Although 
this was not anticipated, it was not entirely a surprise. At that time, the 
Passport Office had only long-term plans to address the general issue of rising 
volumes and their impact on customer service. 


3.70 The field offices did have some measures or techniques that they used 
or developed in response to their volume crisis, but they were ultimately on 
their own. Regional managers and staff informed us that they worked longer 
hours or implemented innovative measures to try to cope with extreme 
volumes, but there was limited assistance offered by headquarters. There was 
no contingency plan, emergency fund, or temporary service locations to deal 
with an unprecedented demand. The Passport Office did not have an 
integrated risk management approach to ensure that it would be able to react 
to emergencies and cope with service demands before those emergencies 
became service failures. 


Inability to forecast and influence demand places an undue burden on service 


3.71 Year after year, demand for passports reaches a peak between 
November and March. As well, during this time there may be busier peak 
days of the week. The Passport Office does not have complete control over 
this phenomenon. However, it can influence demand to some degree. As 
early as its 2000-02 business plan, it discussed the concept of “flattening the 
demand cycle.” The Passport Office’s ability to influence demand is central to 
some of its key plans such as passport on-line and its renewals project, which 
times renewals for favourable periods of the year. At the time of our audit, 
phase I of passport on-line was being offered to the public as of the end of 
January 2005, and the renewal project had not been implemented. 


3.72 In 2002-03, about 2.2 million passports were issued in Canada, a 

13 percent increase over the previous year. The demand for passports 
normally varies according to the strength of the international economy, the 
season, and the occurrence of conflict abroad. But the 2003—04 volume 
significantly exceeded the Passport Office’s forecasts. Forecasting is an 
important activity that is used to set budgets for each regional office. 
Acknowledging that their model required updating, the Passport Office 
developed and approved a new model in 2004. 


3.73 Services are provided through members of Parliament. For over 
30 years, the Passport Office has been accepting passport applications from 
members of Parliament (MPs) who forward them on behalf of their 
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constituents. There is no formal agreement governing the process and the 
Office has not analyzed the security issues presented by this method of 
delivery. In 2003-04, the Passport Office processed 65,700 such applications. 
The service standard is 20 days, which is the same as for a regular application 
received by mail. MPs can enquire about the status of their applications by 
e-mail or a special telephone line. The passports are delivered to the MPs’ 
offices by messenger service or can be mailed back to the applicants. The 
Passport Office estimates the cost per passport for this centralized service is 
less than the mail-in service, because there are fewer errors on the 
applications sent through an MP’s office. 


374 The Passport Office has not analyzed which of its different delivery 
methods are more desirable. It needs to balance the issues of access, security, 
cost, and impact on service delivery. Certain methods of issuing passports may 
be more cost-effective than others. The Passport Office needs to involve 
clients in setting reasonable standards that it can meet at reasonable cost 
without compromising passport integrity. 


3.75 Recommendation. The Passport Office should 


* develop and report on additional service standards, including 
information services such as e-mail, other correspondence, and 
telephone and track and control related costs; 


- develop risk management practices and short-term plans to deal with 
extreme fluctuations in demand for service; and 


* examine its service delivery methods with a view to influencing demand 
while improving cost-effectiveness and enhancing security measures. 


Passport Office’s response. The Passport Office agrees with the 
recommendation. New performance standards were introduced after 
September 11, 2001, as a result of the sudden increase in application volume 
and the introduction of new security requirements. It will develop and report 
on additional service standards. 


It will introduce additional measures in 2005-06 to improve service in the 
areas of correspondence (including e-mail) and telephone inquiries. It will 
also introduce risk management practices and short-term plans to deal with 
fluctuations in volumes from both financial and operational perspectives. 


3.76 Although the Passport Office has documented its strategic planning 
process, weaknesses noted in the security and identity authentication 
activities, together with maintaining an adequate service level at a reasonable 
cost, impose considerable challenges. We examined its readiness to face those 


challenges. 


3.77. We found nine strategic initiatives under way to either improve 
security or service, plus another five important projects that were not listed in 
the corporate planning document. For example, one project underway that 
would enhance security was a facial recognition capacity using submitted 
photos. Another project would streamline the passport renewal process and 
improve service. Many of these projects have been ongoing for several years 
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and are not yet completed. The Passport Office’s management has not 
prioritized those projects and weighed their relative importance. We found 
that the Office used no formal criteria to separate priority from non-priority 
projects, for ranking projects according to greater or lesser importance, or for 
sequencing projects in the order they must take place. Only budget criteria 
were consistently applied in assessing projects—to ensure that a project was 
not too costly. 


3.78 We observed that management had not fully analyzed the impact of 
concurrent improvement projects on each other. For example, the Passport 
Office decided to open several new offices in recent years. At the same time, 
it has had projects in development for applying for a passport on-line and to 
allow people to renew their passports without presenting their source 
documents at passport offices. The Office opened two new offices in the 
Greater Toronto Area assuming they could downsize again if volume dropped 
as a result of those other strategic initiatives. We found that the quality of 
analysis for setting up new offices was poor and did not address many of the 
key requirements of a business case: cost benefit analysis; key assumptions 
and constraints; or implications on future operational costs. 


3.79 Project management and policy change initiatives have a direct impact 
on the jobs of Passport Office staff. For example, the projects on renewals and 
passports on-line may result in a reduction of front-counter staff in local 
offices. On the other hand, the planned requirement for first-time applicants 
to make an appearance, instead of mailing their applications, could increase 
the demand on front-counter staff. We found no long-term, consolidated 
human resources plan to address the impact of such initiatives. 


3.80 In 2002, the Passport Office adopted a framework for human resources 
planning and approved the Human Resources Strategic Plan and Action Plan 
in 2003. Responsibility and timing for several key actions still remain to be 
determined after two years. The Passport Office has yet to address the gap 
between future passport demand and its current human resources capacity. 


3.81 Many technological and quality assurance improvements will need to 
be made to increase the security around the issuing of passports. As well, the 
Passport Office plans to make several changes to service delivery. In our view, 
even considering the improvements to service that have been made, its 
current management systems and practices are not adequate to meet those 
challenges. 


3.82. Recommendation. The Passport Office should review, complete, and 
implement integrated human resource plans without delay and improve its 
management systems to enable it to 


* prioritize its many improvement initiatives by ranking them according to 
approved criteria, and 


* integrate the planning of new offices into its strategic framework and 
consider the mid- to long-term impact of all improvement initiatives on 


each other. 
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Passport Office’s response. The Passport Office agrees to review, complete, 
and implement an integrated human resource plan in light of the Public 
Service Modernization Act. The reorganization and restructuring of the 
Passport Office and the establishment of a Major Projects Bureau in 
February 2005 will allow it to prioritize its improvement initiatives and rank 
them according to approved project management criteria. Improved project 
management processes will have a positive impact in supporting the long- 
term strategic vision of the Passport Office as a whole. The Office will ensure 
that the opening of any new offices will be integrated into the planning 
framework and reflect the strategic direction of the organization. 


3.83 Charging for services differs from taxation in that the charge is for a 
product or service received by an individual, over and above what the general 
taxpayer receives. The Passport Office charged $158 million in fees in 
2003-04 and collected another $54 million in consular fees that were 
deposited to the Conselidated Revenue Fund. Neither the Department of 
Foreign Affairs Canada nor the Passport Office has authority to re-spend the 
$54 million in consular fees. The Passport Office ended the 2003-04 fiscal 
year with a net income for the year of about $9 million and an accumulated 
surplus of $37 million. 


3.84 The passport fee was set at $35 in 1992. In 1995, a consular fee of 
$25 was introduced to be charged on issuance of a passport, bringing the total 
fee to $60 charged to the applicant. In December 2001, the government 
raised the passport fee by $25 to a total of $85 for an adult regular passport. — 
Of this, $60 funds the operations of the Passport Office through its revolving 
fund, and the $25 consular fee recovers consular services costs. The Passport 
Office does not manage or control consular activities in missions abroad. 


3.85 The setting of passport fees is controlled by regulation. Passport fees 
are established by regulation under the Financial Administration Act and can 
only be changed by a regulation. The Act forbids user fee recovery in excess 
of costs. The Federal Regulatory Policy governs the development of 
regulations. 


3.86 Key provisions of the regulatory process require a regulatory impact 
analysis as well as pre-publication in the Canada Gazette of the decision to 
impose a new fee. Prior to changing a fee, the proposal must include an 
analysis of the proposed regulation, the alternatives considered, a benefit-cost 
analysis, the results of consultations with stakeholders, the department or 
agency’s response to any concerns raised, and the means of monitoring and 
enforcing the proposed fee or fee increase. The results of this analysis are 
summarized in a regulatory impact analysis statement. 


3.87 Under the Treasury Board’s Cost Recovery and Charging Policy and 
the subsequent External Charging Policy in effect until November 2004, 
departments had to ensure that there was a process for managing fee disputes. 
Although the Passport Office had an ombudsman function for passport 
issuance, it did not have a fee dispute management process as required by the 


policy. 
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3.88 The 2001 fee increase did not fully comply with authorities The 
Treasury Board’s Guide to the Costing of Outputs in the Government of 
Canada defines full cost as the sum of all incurred costs, direct and indirect. 
The Guide provides examples of incurred costs, which are expenses that have 
been paid for. The Guide is silent about how to set fees based on projected 
future costs. 


3.89 The Treasury Board’s Guide to Costing of Service Delivery for Service 
Standards also states that costs recovered through fees are incurred costs. It 
does not consider using future cost projections as the basis for current fees. 
Passport Office and Treasury Board of Canada Secretariat officials told us that 
cost projections may be valid for the Passport Office as a basis for fee 
determination; they referred to a principle that applies to revolving funds of 
breaking even over a three- to five-year business cycle. We accept as 
reasonable that fees may be set based on projections within such a cycle, to 
the extent they are set to recover only expected incurred costs. This implies 
that, according to the Financial Administration Act, fees would need to be 
lowered if expected costs did not materialize over the cycle. 


3.90 The Treasury Board, on the advice of the Secretariat, recommended 
the approval of the passport fee increase of December 2001 based on its 
projections over a five-year cycle. The projections included the full price of 
future capital project outlays. They also included building an accrual 
accounting surplus of $84 million and a cash surplus of $34 million by 


2006-07. 


3.91 We could find no Treasury Board policies or guidance that would allow 
for setting user fees to include an expected surplus above the projected cost. 


3.92 In 2001, as part of its regular client satisfaction survey, the Passport 
Office asked whether respondents would support a potential $10 or $20 fee 
increase to offset the costs of new services and security features. There was 
low to moderate support. About the same time the fee was increased, service 
standards were lowered significantly. The Office also ran a focus group with 
travel agents and associations where a proposed fee increase of $25 was 
discussed. But there has been little discussion with clients about the 
increasing costs of providing passport services. The Treasury Board policies 
dealing with cost recovery required that departments and agencies identify 
and explain clearly to clients why the services are being delivered in the 
manner they are, how charges are determined, and how costs are being 
controlled. In our view, the Passport Office did not meet this on-going 
requirement. 


3.93 Treasury Board conditions with respect to consular fees were not 
followed. When the Treasury Board approved the introduction of the 
consular fee in 1995, it required the Department of Foreign Affairs and 
International Trade, as it was then called, to absorb future costs and fully 
disclose the costs of consular services in its Main Estimates. However, there 
has been little cost information about consular services in the departmental 
annual reports to Parliament. Foreign Affairs Canada informed us that every 
year it had prepared an analysis of its consular costs. However, we could not 
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audit these analyses for four of the years since 1995-96 because the 
Department could not locate the relevant files. Although the passport 
application showed the consular fee was $25 of the $85, there was no detailed 
information in reports to Parliament about what kind of costs justified the 
$25 fee. Passport applicants were not consulted as required and are not in a 
position to know what they are paying for or how the funds collected relate to 
the operations of consular activity. Consular fees collected increased from 
$34 million in 1996-97 to $54 million in 2003-04. Foreign Affairs Canada 
also directly charged $3 million for miscellaneous consular services in 


2003-04. 


3.94 New user fee legislation imposes additional requirements. As 
explained above, the Passport Office was not fully meeting the requirements 
under the Treasury Board policy on external charging. Parliament passed new 
legislation governing user fees in the spring of 2004. This legislation imposes 
additional requirements for transparency on how costs justify fees and on 
expected service levels. The Passport Office is not ready to comply with the 
new legislation because it lacks reliable cost information related to service 
standards. 


3.95 Recommendation. In consultation with the Treasury Board of Canada 
Secretariat, the Department of Foreign Affairs and the Passport Office should 
review the process for setting the passport fee and client consultation on 
service levels to ensure compliance with the Financial Administration Act, 
user-fee legislation, and Treasury Board policies and guidelines. 


3.96 Recommendation. The Passport Office should 


* develop reliable cost information tied to service standards to justify 
current user fees and any future increases to user fees, and 


* improve its method of client consultation to comply with new user fee 
legislation and ensure it has an appropriate mechanism for resolving fee 
disputes. 


Passport Office’s response. The Passport Office is currently establishing a 
Balanced Scorecard with a series of cost, performance, and security 
indicators. It will be introduced in 2005-06 and fully operational by 2007-08. 
With this approach, the Passport Office will be better able to determine the 
impact of new initiatives on its cost structures and in a position to take the 


appropriate action. 


Activity-based costing will be introduced in 2005-06 and fully operational by 
2007-08. It will allow the Passport Office to align costs with business lines 


and service standards. 


In light of the events of September 11, 2001, the Passport Office did not have 
the opportunity to consult the public on the revised fee increase as security 
measures required immediate implementation. The Passport Office is 
committed to meeting all requirements for consultation in the future. 
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Conclusion 


3.97 The Passport Office is struggling to balance and meet increasing 
security expectations and demands for service. It has not designed and 
implemented quality assurance for the examiner function and for effective 
security controls over the issuing of passports. 


3.98 The Passport Office cannot effectively authenticate an applicant’s 
identity and determine eligibility in all cases. Its watch list is deficient and not 
up-to-date. It has not found ways to automatically obtain necessary data from 
other government sources, or ways to effectively validate identity data with 
the provinces. Management’s monitoring of security controls is poor. The 
Passport Office urgently needs to enter into co-operative arrangements with 
the provinces and other government departments to validate identity data 
and to obtain the data it needs to keep its watch list current and complete. 


3.99 The Passport Office has so far established and achieved transparent 
levels of service, but it has not consistently consulted stakeholders on service 
standards and has had difficulty in controlling its costs. 


3.100 Our audit has highlighted several areas that need urgent attention. 
The Passport Office needs to perform a comprehensive risk assessment of all 
its operations and prepare an action plan. The implementation of this plan 
should be closely monitored. 
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About the Audit 


Objectives 


Our audit objectives were to determine whether the Passport Office achieved the following at a reasonable cost: 
* design, implementation, and monitoring of effective controls over the issuing of passports; and 


* establishment and achievement of reasonable levels of service. 


Scope and approach 


The main focus of our audit was the security measures and identity verification undertaken by Passport Office 
examiners and support staff. We also assessed quality of service to the public, which the Passport Office considers 
integral to its operations. Finally, we assessed whether the Office complied with authorities for setting user fees. 


We examined Foreign Affairs Canada’s financial information about consular activities. We reviewed the 
Department's internal audit reports to assess security and service issues at missions abroad. 


We reviewed regional and headquarters’ documents, interviewed management and staff, interviewed Treasury Board 
of Canada analysts, sampled databases, performed quantitative and qualitative analyses, observed operations and 
conducted interviews at headquarters and 15 offices, and sent an e-mail questionnaire to a sample of examiners. We 
examined a risk-based sample of case files to determine the adequacy of the Passport Office’s operating procedures. 


For our e-mail questionnaire to examiners and our tests of guarantor checks, we selected samples from the 
populations provided to us by the Passport Office. All our samples were taken from the most complete data that were 
available at the time of our audit. We sent an interview questionnaire to 50 randomly selected examiners and 
received 36 responses. For guarantor checks we examined 100 passport applications that required mandatory 
guarantor checks. 


For our examination of the lost and stolen passport reports, we randomly selected 115 reports from daily 
reports beginning in February 2004 when the Passport Office assumed responsibility for entering its data into 
Canadian Police Information Centre database. 


Criteria 
The criteria for this audit were based on the Canadian Passport Order, Passport Services Fees Regulations, the 
Financial Administration Act, Treasury Board policies and guides, and the Passport Office’s manuals and guides. 
We expected the Passport Office to have 
* adequate procedures and processes in place to ensure that passports are only issued to eligible Canadian 
citizens; 
* adequate procedures and processes in place to ensure that reasonable levels of service are established, achieved, 
and reported on; 
* followed a formal project management process to improve service delivery; and 


* complied with authorities for setting user fees. 


Report of the Auditor General of Canada—April 2005 Chapter 3 25 


PASSPORT OFFICE—-PASSPORT SERVICES 


Audit team 
Assistant Auditor General: Shahid Minto 


Principals: Paul Morse, Louis Lalonde 
Directors: Sue Morgan, Robert Cook 


Dusan Duvnjak 
Anthony Levita 
Lyane Maisonneuve 
Jennifer McLeod 
Heather Miller 
Bridget O’Grady 
Nicolas Paré 

Benoit St-Jean 


For information, please contact Communications at (613) 995-3708 or 1-888-761-5953 (toll-free). 
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Chapter 


National Defence 


C4ISR Initiative in Support of 
Command and Control 


All of the audit work in this chapter was conducted in accordance with the standards for assurance engagements set by the 
Canadian Institute of Chartered Accountants. While the Office adopts these standards as the minimum requirement for our audits, 
we also draw upon the standards and practices of other disciplines. 
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Main Points 


Introduction 


Focus of the audit 


Observations and Recommendations 
National Defence is moving in the right direction 


Affordability 


Funding pressures could put some C4ISR projects at risk 


C4ISR Implementation Strategy 


C4ISR systems have been developed without a joint C4ISR doctrine or concept of operations in place 
Interoperability is not yet defined for C4ISR 

Some important elements of a C4ISR implementation strategy need to be better developed 

Stronger C4ISR management is needed | 

Work is starting on a C4ISR enterprise architecture framework 


Use of spiral development methodology is appropriate but needs to advance 


Human resources management 


Action to develop skilled C4ISR human resources is lagging behind other activities 


Conclusion 


About the Audit 
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National Defence 


C4ISR Initiative in Support of 
Command and Control 


Main Points 


41 National Defence estimates that by 2015 it will have invested almost 
$10 billion on projects to improve the way it gathers, processes, and uses 
military information. This is needed to provide commanders with better 
information for decision making in order to exercise faster and more effective 
command and control in both joint and combined operations. It is also to 
allow National Defence to keep up with the progress and changes being made 
by allies. This is a key part of the transformation of the Canadian Forces into 
the 21st century. The Command, Control, Communications, Computers 
(C4), Intelligence, Surveillance, and Reconnaissance (ISR) initiative is a 
strategy to help make this information-technology transformation happen. 


42 The Departrnent has made a good start in managing C4ISR and has 
put in place some guidance and methodology to help it achieve its goals. 
However, some of the key elements required to ensure successful 
implementation of the C4ISR initiative are not yet in place. National 
Defence must put a priority on producing its joint C4ISR doctrine, a concept 
of operations, a clear definition of interoperability, and a common 
understanding of what C4ISR means to better guide its development. 
National Defence has already invested about $4 billion of the almost 

$10 billion planned for projects that have a C4ISR component; however, 
without these key elements the Department is at risk of developing non- 
compatible or duplicate systems. 


43 C4ISR is complex and expensive and constitutes a significant 
investment by the Department in improving the way it collects, processes, 
and uses information for operations, in particular joint and combined 
operations. The Department recognizes that the individual services cannot 
afford to pursue, nor should they pursue, C4ISR separately. The Navy, Army, 
and Air Force must work jointly to take advantage of economies of scale and 
achieve fully interoperable and integrated systems. While the Deputy Chief of 
Defence Staff is responsible for joint operations, the individual services have 
traditionally been responsible for generating what they need for those 
operations; therefore, C4]SR-related development has tended to be pursued 
along individual service paths rather than jointly. As well, existing joint- 
requirements committees need clearer authority and a stronger role in project 
approval to ensure that activities co-ordinate and meet joint expectations. 


44 To have a C4ISR initiative that is affordable and achievable, National 
Defence needs mechanisms that help it choose which projects to pursue and 
to guide how systems will be developed. The Department is using a 
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methodology that helps it define and refine C4ISR needs over time and has 
begun working on a common approach to designing its systems to meet those 
needs. However, projects already under way or being developed have not 
followed a common design approach and need to be reviewed to ensure they 
still meet the Department’s intent. Without this adherence to a common 
approach, the Department has no assurance that its systems will converge 
and integrate as planned. Both a common design approach—or enterprise 


architecture—and a review of all projects need to be completed if the 


Department is to move forward in a structured, efficient, and disciplined way. 


Background and other observations 


45 The goal for the Canadian Forces, like other militaries around the 
world, is to improve the way it collects, analyzes, disseminates, and shares 
information gathered through C4ISR means to provide commanders with 
trusted and relevant information for decision making. Militaries are putting 
renewed efforts into C4ISR because of the opportunities information 
technology provides to enhance their own command and control. 


46 In December 2003, the Department of National Defence released its 
C4ISR Command Guidance and Campaign Plan document to provide the 
Canadian Forces with high-level guidance and an integrated approach to 
develop and transform the capability into one that supports Forces-wide 
command and control into the 21st century. 


47 By 2008, the Department wants to achieve its first C4ISR timeline and 
to have completed three overall phases—concept development, 
consolidation of projects and initiatives, and initial transformation; these 
involve the creation of an information-based culture and a network-enabled 
organization. 


48 Our 1994 Report, Chapter 25, Information Technology recommended 
that command and control systems be able to interoperate in joint and 
combined operations, which the Department accepted as a mandatory 
requirement. The Information Technology Infrastructure and the Canadian 
Forces Command and Control Information System were both actively 
addressing these requirements. The Department acknowledged that joint 
interoperability had not received a high priority in these activities until the 
recent past. Nevertheless, our 1996 follow-up chapter concluded that the 
interoperability of command and control communication systems needed 
further development for joint and combined operations. 


49 In our 1998 Report, Chapter 3, Equipping and Modernizing the 
Canadian Forces, we reported that, as far back as 1994, the Army had not 
kept pace with technology to modernize its equipment; this left it vulnerable 
to threats in low-level and mid-level operations. Some of those problems have 
since been corrected, but some have not. For example, National Defence has 
been working on implementing an Army communications system, which was 
to be completed by 2001 and was to be fully interoperable with other 
Canadian Forces command and control systems. By 2004, however, this was 
still not fully operational. 
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The Department has responded. The Department has indicated the action 
it has taken or intends to take to address the recommendations. Its detailed 
response follows each recommendation throughout the chapter. 
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C4ISR—Command and Control, 
Communications, Computing, Intelligence, 
Surveillance, and Reconnaissance consists of 
the doctrine and concepts, the connectivity, the 
information systems, the sensors, and the tools 
required to effectively support Command across 
the entire spectrum of Canadian Forces 
operations through the timely attainment of 
trusted and relevant information. 


Command and control—The exercise of 
authority and direction by a designated 
commander over assigned forces in the 
accomplishment of the force's mission. 

The functions of command and control are 
performed through an arrangement of personnel, 
equipment, communications, facilities, and 
procedures that are employed by a commander 
in planning, directing, co-ordinating, and 
controlling forces in the accomplishment of 

the mission. 
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Introduction 


4.10 A priority of the Department of National Defence and the Canadian 
Forces is to transform the way it conducts operations by taking advantage of 
advances in information technology. The goal is to support this operational 
transformation by developing C4ISR systems that can provide trusted, 
relevant, and timely decision-quality information to commanders. This will 
reduce uncertainty to the point where the commander is confident that the 
decisions being made are the best obtainable in the operational context. 


411 According to National Defence, the conduct and character of war is 
changing. Rapidly integrating technologies are effecting important changes in 
our global society and in how the military operates. Many view the 
application of advanced technologies as a revolution in military affairs. 
Canada’s allies are also pursuing these developments to ensure that they 
maintain modern and interoperable forces into the future. 


412 Part of National Defence’s role is to ensure Canadian sovereignty and 
to contribute to the defence of North America. In this role, National Defence 
sees a need for a greater ability to function jointly. As well, National Defence 
is now often required to work with other government departments and non- 
governmental organizations. In DND’s view, our allies in NORAD and 
NATO want National Defence to be a competent partner capable of taking a 
meaningful part in combined operations. Therefore, the Canadian Forces 
must be interoperable with its defence partners; this means that it needs to 
keep pace with new military concepts, doctrine, and technological change. 


413 Shaping the Future of the Canadian Forces: A Strategy for 2020 provides 
the official strategic vision for the development of the Canadian Forces in the 
21st Century. In its strategy, the Department states 


At its core, the strategy is to position the force structure of the 
Canadian Forces (CF) to provide Canada with modern, task- 
tailored, and globally deployable combat-capable forces that 
can respond quickly to crises at home and abroad, in joint or 
combined operations. 


414 How well the strategy goals can be achieved will be determined in part 
by how well National Defence can integrate advanced information 
technologies with appropriate operational and organizational concepts. 


4.15 In July 2002, National Defence reviewed its capabilities and 
identified gaps in its five capability areas in the document, Capability 
Outlook 2002-2012. It determined that a very serious shortfall existed in 
intelligence and information for command and control and that it needed to 
better develop joint doctrine, operational concepts, and training. The 
Department recognized that initiatives should be integrated and that failure 
to do so could result in a lack of interoperability within the Canadian Forces 
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and with allies. The Capability Outlook 2002-2012 set out as priorities 
the following: 


* the development of a truly CF-wide joint command system; 


* a comprehensive plan for joint policy, concepts, and doctrine 
development; 


* the development of a comprehensive strategic-operational joint 
command structure; 


* the procurement of a coherent and fully integrated suite of intelligence, 
surveillance, and reconnaissance assets; and 


* the establishment of intelligence and information fusion centres. 


416 The Department has begun its C4ISR initiative to take advantage of 
information technology opportunities and address its capability deficiencies in 
a strategic way but also to ensure that operational requirements of the Navy, 
Army, and Air Force are met. 


417. In its C4ISR capability analysis, the Department identified several 
areas to improve, such as the following: 


* underutilization of existing network capabilities, 
* no common goals or standards, 

¢ lack of integration between projects, 

* lack of a management system, 

¢ need for cultural and organizational change, 

* need for more personnel and training, 

¢ bandwidth constraints, and 


* funding constraints. 


Focus of the audit 


418 We evaluated the Department’s progress in implementing key 
organizational and operational changes that would allow it to build toward 
C4ISR goals. Specifically, we focussed on progress that the Department has 
made in developing C4ISR guidance at the strategic level, which includes 
establishing clear and jointly accepted definitions, doctrine, lines of authority 
and accountability, a framework for a department-wide C4ISR architecture, 
and performance measures. 


4.19 We also examined the progress National Defence has made in bringing 
together work already started to address capability deficiencies through 
C4ISR-proposed solutions. We examined the expected costs of implementing 
C4ISR solutions to determine whether the Department is putting forward 
requirements that are affordable and achievable. More information is 
available in About the Audit at the end of the chapter. 
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Observations and Recommendations 


National Defence is moving in the right direction 


4.20 We found that the Department has made progress toward addressing 
its C4ISR capability gaps. Under the Vice Chief of Defence Staff and the 
Deputy Chief of Defence Staff (DCDS), National Defence has set up the 
Joint Capability Requirement Board and the C4ISR Oversight Committee to 
provide a strategic perspective and leadership. These committees give senior 
management a forum to discuss joint C4ISR initiatives. 


4.21. The Department has also initiated a Joint Capabilities Assessment 
Team to review identified operational requirements and assess whether 
specific projects and initiatives are consistent with joint goals. This 
Assessment Team can then advise the Oversight Committee and the 
Requirement Board on C4ISR opportunities. 


4.22 The Department has produced its C4ISR Command Guidance and 
Campaign Plan as a vision and guidance for achieving C4ISR goals. In the 
Campaign Plan, the Department states that the services (Navy, Army, and 
Air Force) must not independently develop information systems for 
command and control. To do so could result in the services taking divergent 
paths, developing duplicate systems, and failing to take advantage of 
efficiencies. This would increase the risk that systems would not be able to 
interconnect when they need to, which is contrary to the Canadian Forces 


C4ISR goals. 


4.23 ‘To prevent this, the Department has adopted an enterprise model that 
brings together all information management activities and investments as 
part of a larger Defence strategy. Under this model, the services and groups 
are getting out of the business of doing their own information management 
work. The three services and the groups are responsible for identifying and 
determining their requirements and the Assistant Deputy Minister 
(Information Management) is responsible for systems design. By adopting this 
approach, National Defence wants to ensure that common protocols are 
followed as systems are developed. The Department expects that this will 
provide structure and discipline for C4ISR initiatives, so that by 2008 the 
Department will have integrated systems across functional areas. 


Funding pressures could put some C4ISR projects at risk 


4.24 We reviewed the planned spending for 91 projects that National 
Defence identified as a baseline for planning and implementing its C4ISR 
transformation. We estimate that expenditures related to C4ISR could total 
approximately $9.7 billion, of which about $4 billion has been expended to 
date and a further $5.7 billion in spending is planned by the Department over 
the next 10 years. In some years, annual spending for C4ISR will reach as 
much as 40 percent of the funding available in the capital equipment budget. 
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4.25 Exhibit 4.1 shows the forecast spending requirements for C4ISR 
projects and the total demand that this places on the available capital 
equipment budget. Even though the Department has had to make some hard 
choices in allocating its capital budget, more spending is forecast to complete 
already existing projects plus start projects identified in the Strategic 
Capability Investment Plan than the funding available in the capital 
equipment budget. This funding pressure could result in some C4ISR projects 
being scaled back, deferred, or cancelled. 


Exhibit 4.1 C4ISR budget requirements for National Defence 


($ millions) 
3,000 


Strategic Capability Investment Plan and approved projects 
2,500 


2,000 
Nea ital equipment budget 
1,500 pital equip g 


1,000 Total C4ISR approved and planned projects 


Total C4ISR approved projects 


2005-06 2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 


500 


Source: National Defence 


4.26 C4ISR goals and timing could be at risk if the right projects cannot be 
funded or are deferred. The Department recommended a review of its C4ISR 
projects to ensure they are still operationally required, but we found that this 
had not yet been undertaken. This review is necessary to ensure that projects 
that are planned or already under way adhere to C4ISR guidance to avoid 
unnecessary duplication and to obtain the best result. 


427 In its Strategic Capability Investment Plan 2004, National Defence 
identified which C4ISR projects it intends to start as a priority. There are 
three types of C4ISR projects that the Department estimates will need about 
$5.7 billion in funding. 


* Enabler projects. These are considered key to implementing C4ISR. For 
example, the Canadian Forces Command System Phase II project 
($102 million) is key to enabling better information integration. In all, 
the Department plans to spend about $882 million on 13 enabler 
projects over the next 10 years. All enabler projects have already been 
approved or are priorities for funding. 


* Related projects. These are part of the C4ISR initiative to address 
capability deficiencies and continue the C4ISR transformation. For 
example, the Joint Space Support project and the Tactical Information 
Distribution Enhancement project are related projects. The Department 
plans to spend about $1.7 billion on related projects over 
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Doctrine—The set of fundamental principles 
that guide military actions in support of 
objectives. 


Concept of operations—A clear and concise 
statement of the line of action chosen by a 
commander in order to accomplish the mission. 
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the next 10 years. We found that of 19 related projects, 10 have already 
been approved and are underway or completed. The other nine related 
projects, if approved, will require about $1.25 billion in funding over the 
next 10 years. 


* Other projects identified as part of C4ISR. There are 59 other C4ISR 
projects that the Department has planned or already started. The 
Department has identified a need for about $3.1 billion in ongoing 
spending for these projects over the next 10 years. 


C4ISR systems have been developed without a joint C4ISR doctrine or 
concept of operations in place 


4.28 ‘To be operationally relevant, defence programs are based on supporting 
doctrine and concepts of operations. A concept of operations guides 
planning, which translates strategic objectives into military operations by 
clearly stating the action needed to accomplish the mission. Doctrine is 
essential as it sets out the fundamental principles for how military operations 
are to be planned, trade-offs addressed, and actions conducted. The 
Canadian Forces Command System Framework Concept of Operations states 
that any equipment program, which includes C4ISR projects, must be based 
on supporting doctrine. 


4.29. The C4ISR Command Guidance and Campaign Plan states that 
doctrine, concepts, and systems need to evolve along a single, convergent 
path that is interoperable across all environments. Without these, the 
Canadian Forces would not have a blueprint for how it intends to pursue 
C4ISR development to support the operational demands of the three services 
in a common way. 


430 We found that the 91 C4ISR projects are proceeding without this 
blueprint in place. Therefore, it is difficult to link the projects back to an 
overall joint C4ISR objective or to obtain assurance that C4ISR work that is 
planned or already under way is proceeding in a common, interoperable way. 


431 The three separate services—the Navy, Army, and Air Force—are 
responsible for developing and maintaining their own concepts of operations 
and doctrine. The Deputy Chief of the Defence Staff is responsible for 
doctrine and concepts of operations for joint and combined operations. The 
Department has stated that it has not yet bridged the gap between the joint 
level and the service level doctrine. 


4.32 In August 2002, the Department noted that no effort was under way 
for the development of joint doctrine for C4ISR. At the time of this audit, the 
Department still had no plans to develop C4ISR doctrine even though the 
Defence Plan had called for joint interoperability doctrine and procedures to 
be developed and implemented by July 2004. 


4.33 In order to develop systems consistent with its C4ISR vision, the 
Department needs to have doctrine and a concept of operations in place to 
ensure that it is not going in the direction it had warned itself against—that 
is, the development of duplicate or incompatible systems. With planned 
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Interoperability—tThe Canadian Forces does 
not have a definition of interoperability. 

NATO defines interoperability as the ability of 
Alliance forces and, when appropriate, forces of 
Partner and other nations to train, exercise, and 
operate effectively together in the execution of 
assigned missions and tasks. 


Self-synchronization—The alignment of 
Groups’ and Services’ activities, to coincide with 
and complement the overall Campaign Plan, 


without external control. 


C4ISR expenditures totalling almost $6 billion over the next 10 years, these 
key elements should be in place as soon as possible. 


4.34 ‘Tactical data links and bandwidth requirements are two examples 
of C4ISR projects that need a C4ISR concept of operation to guide 
development (see Tactical data links and bandwidth requirements on page 11). 


Interoperability is not yet defined for C41SR 


4.35 National Defence has also expressed concern over its current level of 
doctrinal interoperability with its allies. An assessment conducted in 2002 
concluded that most interoperability concerns could have a major or 
significant impact on operations. 


436 National Defence does not have an approved definition of 
interoperability for joint and combined operations. It has not adopted the 
NATO definition of interoperability for joint and combined operations into 
its documentation. According to departmental documents, the NATO 
definition is NATO-centric and does not suit Canadian Forces requirements. 
However, no other definition has been developed. The Department plans to 
establish an interoperability working group to address the need for a 
departmental interoperability plan. 


Some important elements of a C4ISR implementation strategy need to be better developed 


437 Tasking. The Defence Plans make it clear that National Defence will 
develop its C4ISR capability; further, the Plans give specific direction to the 
DCDS, Chief of the Maritime Staff, Chief of the Land Staff, Chief of the Air 
Staff, ADM (Information Management), and ADM (Science and 
Technology) to do so. The services and groups are to “self-synchronize” their 
activities and efforts using the C4ISR Command Guidance and Campaign 
Plan and the Target Integration Model 2008 increments. 


438 ‘This self-synchronization is to occur through the business planning 
process, in which the three services and the groups identify their operational 
requirements and any related C4ISR needs. The services and groups need to 
determine how they will self-synchronize their activities. However, the C4ISR 
Command Guidance and Campaign Plan explains that self-synchronization 
will take place as synergy is created between C4ISR doctrine, concepts, and 
capabilities. Therefore, in order for selfsynchronization to work, the 
Department needs to have in place the necessary doctrine and concepts of 
operations. 


439 Directives. The C4ISR Command Guidance and Campaign Plan was 
approved by the C4ISR Oversight Committee and signed by the DCDS as the 
Chair of the C41SR Oversight Committee. A presentation was made to the 
Defence Management Committee on the progress achieved so far, in 
February 2003, and the Requirement Board endorsed the Campaign Plan in 
December 2003. However, we did not find any directive issued by the Deputy 
Minister or the Chief of the Defence Staff to implement the C4ISR 
Command Guidance and Campaign Plan. Nor did we find any directives 
issued by groups and services to their subordinate levels. 
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Tactical data links and bandwidth requirements 


The C4ISR initiative is intended to reduce uncertainty to the point where commanders 
are confident that the decisions they are making are the best obtainable in the 
operational circumstances. The overall vision is to develop a C4ISR capability that 
supports the command and control function by providing commanders with situational 
awareness and a common operating picture in real or near-real time. Investment in 
CAISR will enable networked forces, tailored by task, to conduct integrated operations 
across the full range of military activities. 


Tactical data links. Tactical data links (TDL) are a technology that enables 
standardized communication pathways to commanders during operations. TDLs 
connect commanders in networks to a continuous flow of information, so that each 
commander can then make best use of available sensors and weapons. 


Tactical data links are primarily for wireless transmission via radio and satellite, but 
connections through secure landline are also vital to support deployed operations. TDL 
capabilities have been identified as a “must have” for coalition operations. The Chief of 
the Defence Staff has also underscored the need for joint force development and to 
properly situate TDL development within the Department. However, there is a high risk 
that TDL capabilities will be implemented without a consolidated joint strategy across 
the Canadian Forces. There is no one source in the Canadian Forces that provides an 
overview of wireless systems employed or planned for use in the Canadian Forces. A 
Concept of Operations for Tactical Data Link Systems was drafted in February 2003 
but has not yet been approved. 


In 2004, there are three projects to develop TDL capability within the Navy and the 
Air Force, estimated at $175 million, of which $25 million has already been spent by 
the Department. 


Bandwidth. Bandwidth refers to the size of the connecting “pipe” that information 
flows through between communication terminals. Bandwidth determines the rate at 
which data can be transmitted: the greater the bandwidth, the greater the amount of 
information that can be sent to networked commanders in a given amount of time. 
Sufficient secure bandwidth must be provided to carry out C4ISR functions. 


The Canadian Forces’ bandwidth use has increased eight-fold in eight years and is 
expected to continue to increase due to the demands of new sensor data, information 
systems, and ISR fusion requirements. By 2010, unless changes are made, National 
Defence expects it will not be able to meet bandwidth requirements. The Department 
has endorsed a strategy to develop capability to respond to the need to provide 
sufficient bandwidth to deployed operations. The endorsed strategy emphasizes 
increasing joint investments in satellite technology. 


Currently, however, there is no common concept of operations to manage satellite 
channels and bandwidth demand within the Department. The capabilities for 
managing channels that exist have been developed on a system-by-system basis. No 
standardized architecture or commonly recognized interface standard exists to address 
needs for increasing bandwidth capacity over satellite data links for command and 
control. Without this architecture, there could be a proliferation of independent 
communication systems instead of networked systems. 


In 2003, the Department spent $18.7 million to lease commercia! satellite capacity to 
support deployed operations. Currently, there are five projects in the Department to 
develop satellite broadcast capabilities and improve joint satellite communications, 
estimated at $693 million. 
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440 The Department may issue a departmental administrative order and 
directive to announce the departmental policy to implement the C4ISR 
initiative, but this has not been done. Such an administrative order could 
provide specifications on definitions, policy direction, roles, responsibilities, and 
authority. 


4.41 Definition. We found that different people interpret C4ISR to mean 
different things; the Department also commented on this issue in its 
documentation. The C4ISR Command Guidance and Campaign Plan needs 
to be updated to include achievable goals, criteria to declare success, mission, 
resources, constraints, and strategic direction to commanders as outlined in 
the Canadian Forces Operational Planning Process. If this is done, it would 


better clarify C4ISR. 


4.42 Groups and services plans. Consistent with the Department's 
enterprise model, in March 2004 the Assistant Deputy Minister (Information 
Management) requested that groups and services produce an information 
management/information technology plan by June 2004 to interface with the 
information management group. However, at the time of our audit, ADM 
(IM) had not received all of the expected information. 


4.43 Supporting plans. We found that of the ten supporting plans identified 
in the C4ISR Command Guidance and Campaign Plan, work had started on 
only four. Work on the ten plans was to have been initiated by April 2004. 
Although departmental documentation indicates that four plans were 
developed, the Department was not able to provide us with copies of any of 
the plans. 


4.44 C4ISR implementation risk assessment. National Defence has an 
Integrated Strategic Risk Management Framework to define corporate risk 
and how to manage it. The framework is intended to suggest risk- 
management actions and methods that will support Defence objectives. The 
Treasury Board concept for an integrated risk-management function expects 
that departments will set up the corporate infrastructure for risk management 
to provide clear direction and senior-management support. National Defence 
has chosen to depart from the Treasury Board approach and use its existing 
committee structure and planning process. 


445 The Department has not yet conducted a risk assessment for its 
strategic C4ISR initiative. The C4ISR Command Guidance and Campaign 
Plan identified “risk mitigation decisive points,” which talk about five possible 
risk weaknesses. However, the Department needs to fully assess these areas 
against risk likelihood and potential impact and then develop a risk- 
management strategy. 


4.46 Recommendation. National Defence should put a priority on 
developing joint C4ISR doctrine, concept of operations, and a definition of 
interoperability by the end of 2006. 
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Department’s response. Agreed. Doctrine, operating concepts, and clear 
definitions are recognized as important elements of C4ISR implementation, 
and their development continues to be a high priority. Although the writing 
and approval of doctrine and concepts of operations are complex 
undertakings, the Department will endeavour to have these elements 


developed by the end of 2006. 


Stronger C4ISR management is needed 


447 The C4ISR Oversight Committee and the Joint Capability 
Requirement Board were created in 1999 and 2000 respectively in response 
to a need for more co-ordination of requirements. The Joint Capability 
Requirement Board’s responsibility is to facilitate a joint understanding of 
concepts of employment and operations, reach consensus for statements of 
operational requirements, and resolve issues of project scope at the corporate 
level. The C4ISR Oversight Committee provides a strategic perspective and 
leadership on all C41SR—related matters. 


4.48 Departmental documents indicate that there are problems with this 
committee structure due to a lack of clear lines of authority. Although the 
C4ISR Oversight Committee is in place to provide executive oversight and a 
co-ordinating role to manage the ongoing tasking of C4ISR, there is no one 
C4ISR authority enforcing important elements such as the implementation of 
the C4ISR Command Guidance and Campaign Plan or the review of projects. 


4.49 We found that departmental officials often bypassed this committee 
structure prior to C4ISR projects approval. Therefore, we were unable to 
determine how these committees were able to fulfill their review and 
co-ordinating role for C4ISR. Since 1999, only 6 of 13 enabler projects and 
2 of 19 related projects were submitted to either the C4ISR Oversight 
Committee or Requirement Board for discussion before going to the Program 
Management Board for funding approval. 


450 We also found that about 72 percent of the C4ISR enabler and related 
projects were missing key documentation necessary for project review and 
approval. Projects were missing either the synopsis sheet to identify 
deficiencies or the statement of operational requirements, or both. To date, 
National Defence has spent about $2.9 billion, of the estimated $3.7 billion 
required, on enabler and related projects that did not have approved 
statements of capability deficiency or statements of operational requirement. 


451 The Joint Capabilities Assessment Team noted that it was very difficult 
to create a list of prioritized C4ISR projects because such a process requires 
the comparison of a large number of dissimilar items. The Assessment Team 
also noted that the problem is further complicated by the breadth of issues 
being considered, which are so large that it is difficult to find a decision- 
making body with detailed knowledge of all projects. 
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Enterprise architecture—A strategic 
information asset base, which defines the 
mission, the information necessary to perform 
the mission, the technologies necessary to 
perform the mission, and the transitional 
processes for implementing new technologies in 
response to changing mission needs. 


Portfolios—tThe military portfolio 
encompasses three generic system families— 
Command Decision Support, Sensor, and 
Weapon (control) system families. 

The corporate portfolio encompasses one 
generic system family—the resources 
management systems family. 

The common portfolio encompasses two generic 
system families—the information management 
and security system families. 


452 Department officials informed us that the Requirement Board and 
C4ISR Oversight Committee do not have the resources to become more 
involved in the project selection process as much more staff effort would be 
required. As well, the Oversight Committee would have to meet more 
frequently. 


453 Recommendation. Senior management should ensure that, in the 
future, all projects justified as part of C4ISR are reviewed and approved by 
the Joint Capability Requirement Board and the C4ISR Oversight 
Committee, as part of the project approval process to provide assurance that 
the projects will ultimately be compatible with C4ISR goals as they evolve. 


Department’s response. Agreed. Future C4ISR projects will be subject to 
review by the Joint Capability Requirement Board and the C4ISR Oversight 
Committee. 


Work is starting on a C4ISR enterprise architecture framework 


454 In 2002, an internal review of information management found that 
National Defence did not have a comprehensive, information-management 
architecture and found several instances of duplication and overlap. As a 
result of these findings, the Department adopted its enterprise model for 
information management, which included a commitment to create an 
enterprise architecture. The Assistant Deputy Minister (Information 
Management) has begun working on an enterprise architecture framework 
that includes developing a plan to show how information will be sourced, 
collected, and integrated within and across functional areas in the 
Department. 


4.55 The Department has identified activities it needs to do to begin 
developing its enterprise architecture. It issued an Integrated Defence 
Enterprise Architecture Plan (Draft)—October 2004 to communicate the 
core enterprise architecture principles of the Department and to plan which 
activities it will complete by 2006. 


4.56 The Department acknowledges that its key challenges in enterprise 
architecture are to determine, identify, and plan which portions of its three 
defence information management portfolios—military, corporate, and 
common—must become interconnected, interoperable, or integrated to 
avoid duplication and increase flexibility. As well, the Department needs to 
devote the appropriate resources to the right priorities. 


457 Another challenge for the Department is to ensure that C4ISR projects 
already planned or currently under way will be aligned with the enterprise 
architecture that is to be developed. The Department needs to take 
appropriate action to ensure that systems will converge the way senior 
management expects. The Department recognized that a review of projects 
and initiatives, developed either centrally or by the separate services, should 
be done since many projects were begun before National Defence issued its 
C4ISR Command Guidance and Campaign Plan. As a result, the Department 
needs to ensure that projects planned or already under way are consistent 
with the direction now being developed. 
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4.58 Without an enterprise architecture, the Department runs the risk of 
buying ‘and building duplicate systems or systems that are incompatible and 
unnecessarily costly to maintain and integrate. This architecture is critical 
because it serves to inform, guide, and constrain the decisions of the 
Department, especially those related to investments in information 
technology. 


4.59 Recommendation. National Defence should complete its enterprise 
architecture as a priority. 


Department’s response. Agreed. Based on the findings of the Department’s 
Information Management Strategic Review, and ongoing activity to establish 
an Information Management Strategic Plan, the establishment of an 
enterprise architecture is indeed a high priority. Enterprise architecture and 
the C4ISR Campaign Plan should build upon each other in an iterative, 
symbiotic manner in order to produce an enterprise architecture that best 
supports operational requirements and, by extension, C4ISR. 


Use of spiral development methodology is appropriate but needs to advance 


460 By 2008, National Defence plans to have implemented C4ISR through 
11 increments of 6 months each, which are known as spirals. These are 
leading up to its Target Integration Model 2008. Each spiral has objectives 
and/or capabilities. At the end of each spiral, the objectives or capabilities 
acquired are used as a base for the achievement of the next spiral. Those not 
met may be deferred to future spirals or cancelled. Such flexibility separates — 
spiral development methodology from the traditional method or waterfall 
development. 


461 Spiral development methodology seems appropriate to C4ISR 
implementation since the overall requirements are broadly defined, and the 
specific end-product is not yet known. This methodology allows for the 
evolution and refinement of requirements as necessary during development. 
At the time of our audit, the Department stated that it had completed the 
first and second spirals and was beginning work on the third. 


462 One of the goals of C4ISR is to support the operational requirements of 
the three Services and Groups. Initially, the Department chose to develop its 
C4ISR plans centrally and, following the release of the C4ISR Command 
Guidance and Campaign Plan, would increase participation. However, the 
Department was unable to provide evidence of user involvement in defining 
C4ISR requirements at the operational level. We were informed that a 
process to solicit feedback does not currently exist; however, the Department 
has recently held a user workshop. Obtaining and assessing user requirements 
is critical to the success of spiral methodology because, without identified user 
requirements, there is a risk that the objectives and capabilities developed 
will not meet the needs of users. 


463 The spiral objectives were developed in the absence of an enterprise 
architecture. We believe that future spirals should be re-aligned to the 
enterprise architecture as it is developed to provide senior management with 
assurance that the C4ISR initiative is going in the right direction. The 
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Department has recognized the need to align C4ISR with an enterprise 
architecture and informed us that ways and means of resourcing this task are 


currently under consideration. 


464 We also noted that the spiral objectives stated by the Department are 
broadly defined and subject to wide interpretation, which makes it easy for 
the Department to conclude they have been achieved. Exhibit 4.2 provides 


our review of some of the objectives stated in spiral two and how they could 


be improved, so that they are more specific, measurable, achievable, relevant, 


and time-related. 


Exhibit 4.2 A review of spiral-two objectives 


Spiral-two objective 


| Long-form description of 


spiral-two objective 


Suggested elements to add to transform to smart objective 


7(h) - 2.21 “LOG COP [Logistics Common ¢ Define “initiated” 
ing Pi ny 
Operate uel n chp tan ¢ List specific outcomes (expected qualitative and quantitative 
with regards to developing Renattes 10 rene leting thi Rel 
<isctainmment iniomnetioncn a enefits) to be gained from completing this analysis 
CF-wide COP is to be initiated. * Describe and define methodology to be used for the analysis or 
It is requested that the Sustain exercise 
JCAT [Joint Capabilities : ie se 
Assessment Team] act as lead ¢ Describe analysis’/exercise’s relevancy to C4ISR 
agency.” * Specify owner (individual’s name) accountable for the completion 
of the analysis or exercise 
¢ Define completion (examples: “draft”; “mature draft”; “final 
approved draft”) 
¢ Specify owner (individual’s name) accountable for the review of 
the analysis/exercise after completion 
¢ Specify resources required 
¢ Specify stakeholder groups to be consulted, if any 
8(a) — 2.22 “CA4ISR CP—R&D (C4ISR ¢ Specify outcomes (expected qualitative and quantitative benefits) 
Campaign Plan—Research & to be gained from completing this analysis 
Development. ; f 
Component: Identify links ¢ Describe exercise relevancy to C4ISR 
between current C4ISR R&D * Describe and define methodology to be used for the gap analysis 
and OR (Operational Research) 
projects and the C4ISR CP ¢ Specify owner (individual's name) accountable for the review of 
and, identify gaps, if any, in the the gap analysis 
current C4ISR R&D/OR * Specify if approval is required for completion to take place 
| program and resulting tasks.” 
¢ Specify resources required 
¢ Specify stakeholder groups to be consulted, if any 
¢ Define whether the 6-month time frame is adequate or whether a 
portion will be deferred to the next spiral. If so, describe pieces 
expected to be deferred 
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Exhibit 4.2 A review of spiral-two objectives (continued) 


Spiral-two objective 


Ofa) = (2:26 
O(c) = 2.28 


Long-form description of 
spiral-two objective 


“Industry Engagement Strategy: 
DJFC (Director Joint Force 
Capabilities) and DBDO 
(Director Business 
Development Office) (DRDC) 
(Defence Research 
Development Canada) to lead 
efforts in developing an 
Industry Engagement Strategy 
for C4ISR development.” 


“National Operations/ 
Intelligence Centre C4ISR 
Policy: DJFC, in conjunction 
with NDCC (National Defence 
Command Center) JOG (Joint 
Operating Group) and JIIFC 
(Joint Information and 
Intelligence Fusion Capability), 
is to lead efforts in establishing 
pervasive procedures and 
policy for common use 
throughout CF Operations/ 
Intelligence Centres.” 


Suggested elements to add to transform to smart objective 


Define “Engagement Strategy” and “pervasive procedures” 


List specific outcomes (expected qualitative and quantitative 
benefits) to be gained from these analyses/exercises 


Describe exercises’ relevancy to C4ISR 


Describe and define methodology to be used for the Strategy and 
pervasive procedures 


Specify owner (individual’s name) accountable for the review of 
the Strategy and pervasive procedures 


Specify if approval is required for completion to take place 
Specify resources required 
Specify stakeholder groups to be consulted, if any 


Define whether the 6-month time frame is adequate or whether a 
portion will be deferred to the next spiral. If so, describe pieces 
expected to be deferred 


10 — 2.30 to 2.35 


“CAISR Supporting Plans: 
Spiral 1 tasked the initiation of 
a series of plans to support the 
C4ISR CP. A number of these 
plans did not achieve maturity 
during Spiral 1.” 


Define “initiation” 


Cite assurances obtained that no other supporting plans are 
required and or relevant for C4ISR purposes, and how assurances 
were obtained 


Specify outcomes (expected qualitative and quantitative benefits) 
to be gained from “initiating” these supporting plans 
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Describe and define of methodology to be used for the “initiation’ 
of the supporting plans 


Specify owner (individual’s name) accountable for the review of 
each supporting plan 


Specify if approval is required for completion to take place 
Specify resources required for each plan 
Specify stakeholder groups to be consulted, if any 


Define whether the 6-month time frame is adequate or whether a 
portion will be deferred to the next spiral. If so, describe pieces 
expected to be deferred. 


Source: National Defence spiral-two objectives 
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Human resources management 


465 We found that key performance indicators had not yet been developed 
at the time of this audit. The absence of indicators means that the 
Department can declare that any or all of its objectives have been achieved. 
For example, the Department had concluded that 20 of the 41 spiral-one 
objectives had been significantly or totally fulfilled, but we were informed that 
this assessment was done subjectively, with no formal assessment criteria. Our 
review found that, for 12 of those 20 objectives, it was difficult to conclude, 
from the evidence presented, whether substantial completion had in fact 
been achieved. It was too early to assess the success of spiral-two objectives, 
as they were just being completed. 


466 Recommendation. The Department should review its spiral objectives 
to ensure that they are sufficiently clear so that results can be measured. 


Department’s response. Agreed. The “spiral” objectives in the C4ISR 
Campaign Plan are under constant review. Spiral development is actually 
designed to evolve as implementation proceeds and the desired end state 
becomes clearer. 


The tasks and objectives analyzed during this audit were in the first two 
spirals of the campaign plan. With the passing of each spiral, lessons are 
learned, tasking methodology is improved, and objectives are clarified. In this 
regard, many of the suggestions made by the audit team during the course of 
this audit have already been incorporated into spiral three and will be built 
increasingly into subsequent spirals. 


467 Recommendation. The Department should develop key performance 
indicators that can provide senior management with a true measure of what 
has been achieved. 


Department’s response. Agreed. As outlined in the response to 4.66 above, 
the Department will establish and implement measurable indicators of 
success, where practicable. 


Action to develop skilled C4ISR human resources is lagging behind other activities 


468 C4ISR transformation will require the appropriate number of personnel 
with the right skill sets to exploit information technology opportunities. In 
2003 the Canadian Forces was short more than 700 officers and enlisted 
members with the necessary skills and training in occupations needed for 
C4ISR. National Defence believes that most skill sets associated with 
developing C4ISR capability can be achieved from within the Canadian 
Forces. However, three critical skills sets have been identified that will be 
difficult to address within available and projected military human resource 
levels: 


* ISR fusion analysts—for specific spectral systems, 


* content managers—for information processing and database 
management, and 


¢ Web administrators—for Web-based content management and 
dissemination. 
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469 Conversion training and modification of existing training plans for the 
necessary skill-sets and career paths will not begin until 2007. As well, 
according to National Defence, a four to seven-year lead time is required to 
recruit and train personnel with strong information management or 
information technology skills. Attrition is expected to add to the shortage of 
C4ISR skills. The Department recognizes that mitigating strategies must be 
developed to address issues such as training and technical support services. 


470 The Department is addressing some concerns through its Military 
Occupational Structure Analysis, Redesign and Tailoring (MOSART) 
project, but we were informed that identification of C4ISR human resources 
requirements will not be finished until fall 2007. The Department needs 
MOSART to determine its C4ISR-job requirements so that it can ensure that 
its C4ISR direction is realistic and achievable, given human resource limits. 
Training and recruiting programs must be developed in time to have skilled 
people available when needed. We are concerned that work to resolve human 
resource issues is not advancing at the same pace as the rest of the C4ISR 
transformation, putting at risk the ability of the Department to meet its own 
demands. 


471 Recommendation. Senior management should take action to more 
fully and quickly integrate human resources planning into its C4ISR 
implementation strategy. 


Department’s response. Agreed. The Department is very cognizant of the 
importance of the human dimension of C4ISR, and we are currently making 
every effort to ensure that appropriately skilled personnel will be available 
when they are required to fill C4ISR jobs. In this regard, although it is 
acknowledged that it does take four to seven years from recruitment until 
technical personnel are fully trained, the majority of C4ISR jobs are expected 
to require senior military personnel. For the near future, therefore, these jobs 
will be filled by existing military personnel rather than by new recruits. Where 
practicable the military personnel selected to fill these C4ISR jobs will 
receive any additional skill sets required for specific jobs. In addition, some 
skill-set shortages can be mitigated through the use of civilian resources. 


In recognition of the importance of HR, the assistant deputy ministers 
responsible for military and civilian personnel are represented on all 
departmental decision-making and review boards. They have also put an HR 
plan in place to address C4ISR requirements. This plan has been fully 
integrated into the overall C4ISR Campaign Plan and is aggressively being 
followed to provide the HR requirements needed to achieve a fully integrated 
and interoperable C4ISR capability by 2008. 
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Conclusion 


472 The C4ISR initiative that the Department is pursuing is the result of its 
assessment of critical command and control capability deficiencies, which it 
recognizes must be addressed. To ensure that it achieves its goal of providing 
commanders with decision-quality information when they need it, the 
Department wants to have fully integrated and interoperable command and 
control systems by 2008. If the Department is to achieve this timeline, there 
are some barriers to C4ISR implementation that it must resolve soon. 


473 National Defence has put in place appropriate spiral methodology for 
identifying and refining requirements, which allows them to make necessary 
changes during development. With this methodology the Department needs 
to do the following activities to improve results: 


* conduct a review of current and planned C4ISR projects, 
* clearly define spiral objectives and performance measures, 
* increase user involvement in defining requirements, and 


* ensure that the development of an enterprise architecture is a priority 
and that C4ISR objectives are consistent with enterprise architecture 
intent. 


474 Funding for C4ISR projects will have to compete with other capital 
budget priorities. The Department has identified priorities and planned its 
capital spending, but plans exceed budget availability. These funding 
pressures put these projects at risk. All projects need to be reviewed to ensure 
that the right projects are prioritized and that they are consistent with the 
stated C4ISR objectives. 


415 Some components of the C4ISR initiative are progressing faster than 
others, and this puts the Department at risk of developing systems that 
duplicate each other or will not be interoperable and integrated. Systems are 
developing without joint C4ISR doctrine or concept of operations, including 
a definition of interoperability, or enterprise architecture in place to ensure 
that they develop in a common way. This system development guidance is 
necessary if systems are to integrate in 2008. As well, efforts to ensure that 
appropriate skilled personnel will be ready when they are needed are lagging. 


476 Governance mechanisms that are guiding C4ISR progress are not 
strong enough. Questions of policy direction and authority have not been 
resolved despite departmental recognition of problems. As a result, some 
projects are not getting needed committee challenge and endorsement, 
uncertainty exists in defining C4ISR objectives, and joint and combined 
interoperability remains unclear. 
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About the Audit 


Objectives 


The overall objective of our audit was to determine how well National Defence has been able to identify and exploit 
information technology opportunities to support command and control and whether National Defence has a plan 
that is affordable and achievable. 


More specifically, we assessed the extent to which 
* the C4ISR concept could support the command and control capability of the Canadian Forces from the 


strategic level down to operational and tactical levels; and 


* the C4ISR plans and projects were achievable and affordable. 


Scope and approach 


We conducted our audit at National Defence Headquarters, and conducted field visits to Maritime Forces Atlantic 
(Halifax) and Maritime Forces Pacific (Esquimalt), Land Forces Québec Area (Montréal), Area Support Unit and 
5th Brigade Group in Valcartier, 1 Canadian Air Division and the Canadian Northern Region headquarters for 
NORAD in Winnipeg as well as Defence Research and Development Canada establishments in Valcartier and 
Shirley’s Bay. ; 


The audit team interviewed personnel in the DCDS and VCDS organizations as well as ADM (IM), ADM (S&T), 
ADM (Mat), and ADM (Fin CS) personnel; examined department files; and relevant documents. NATO and 
NORAD policies, standards, and agreements were referenced where appropriate. 


Expenditures were tracked using data from the Financial Management and Accounting System, the Capability 
Initiatives Data Bank, and the Strategic Capability Investment Plan. 


We looked for a clear understanding of C4ISR concepts and connectivity requirements. We investigated whether 
projects identified as needed were affordable and achievable, and examined interoperability for the conduct of joint 
and combined operations. We also examined how the C4ISR project approval process examines requirements for 
implementation of the C4ISR initiative. 


Criteria 


We expected that National Defence would 


* clearly define C4ISR and articulate what the Department is trying to achieve through objectives that are 
specific, measurable, achievable, relevant, and time-oriented; 


* ensure that the implementation of the C4ISR Campaign Plan is affordable within the planning time frame; and 


* comply with the Treasury Board Secretariat and National Defence risk management policy and framework in 


implementing the C4ISR concept. 
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Rating Selected Departmental 
Performance Reports 


All of the audit work in this chapter was conducted in accordance with the standards for assurance engagements set by the 
Canadian Institute of Chartered Accountants. While the Office adopts these standards as the minimum requirement for our audits, 
we also draw upon the standards and practices of other disciplines. 
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Rating Selected Departmental 
Performance Reports 


Main Points 


5.1 We assessed the quality of the departmental performance reports of 
three departments—Environment Canada, Fisheries and Oceans Canada, 
and Natural Resources Canada—to determine whether progress had been 
made in improving the quality of their reports against our criteria. In assessing 
the departments’ performance reports against our rating model, we found that 
the quality of the reports had improved marginally, at most, over the two-year 
period between 2002-03 and 2003-04. Over the nine-year period, between 
1995-96 and 2003-04, we found that two departments had achieved some 
modest improvements while the other department showed mixed results. 
Despite these modest improvements in performance reporting over the nine 
years, the latest performance reports still fall short of our criteria for good- 
quality reporting. 


5.2 We found that most of the performance reports provide a good 
overview of each department’s organizational context and planned strategic. 
outcomes. However, performance expectations are not always clear and 
concrete, and the information does not necessarily focus on program results. 
Furthermore, the reported results are not always balanced and supported by 
data sources and data limitations. As well, the reports provide little evidence 
that performance information is used to make decisions about improving 
program results in future years. 


5.3. While we cannot generalize from the results of rating three reports in 
each of three years to trends across all government departments, our findings 
continue to leave us as concerned about the overall quality of reporting as we 
were in our previous audits and studies. Based on these findings, it is 
reasonable to suggest that, without greater effort by departments and scrutiny 
by parliamentary committees, any significant improvements in the quality of 
performance reporting may take decades rather than just a few years. In our 
view, this rate of improvement is not good enough for parliamentarians and 
Canadians to be able to hold departments and agencies to account for their 


performance. 


Background and other observations 


5.4 Parliament holds government accountable for spending in the previous 
year, influences future plans and priorities, and approves expenditures for the 
upcoming fiscal year. Parliamentarians should rely on performance reports to 
keep informed about the performance of the government on key issues. They 
should also use these reports to hold ministers to account for departmental 
spending. As well, Canadians want to know if their taxes were spent where 
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parliamentarians intended. They also want to know if they received good 
value from their government in return for their taxes. Consequently, effective 
accountability to parliamentarians and Canadians depends in large part on 
good-quality performance reporting. 


5.5 In response to this need, Parliament and the government have sought 
to improve the quality of departmental performance reporting for some time. 
For example, Part III of the Estimates was created in 1981 and then separated 
into planning and reporting documents on a pilot basis in 1995 and 
permanently in 1997. 


5.6 Our Office has also been concerned about the quality of performance 
reporting. We have reflected this concern through our audits since at 

least 1988 when we found clear shortcomings while also noting some 
improvements. In 2000 we examined the quality of departmental 
performance reports and concluded that federal departments’ and agencies’ 
progress in improving the quality of their performance reporting to Parliament 
was disappointing. 


5.7 To assist us in our subsequent work, we developed a formal set of 
criteria for good reporting and presented it in 2002 as a model for rating 
departmental performance reports. In the following year, we reported on the 
results of applying this rating model to the departmental performance reports 
of nine departments. While we identified some promising practices, together 
with many challenges, overall we found that the reports did not measure up 
against our model as well as we had expected. 


5.8 This is the second time that we have used our model to rate a selection 
of performance reports. However, it is the first time we have compared a 
department’s reports over one or more periods to assess progress in improving 
the quality of reporting. Overall, we found that progress in improving 
performance reporting to Parliament continues to be a challenge for the three 
departments, even after almost a decade of effort. 


The government has responded. The government has responded to our 
observations and informs us that it will continue its efforts to improve 
performance reporting across government departments. 
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Introduction 


Performance information continues to be important 


5.9  Good-quality performance reporting is a key means for Canadians to 
hold their government to account. This is because Canadians want to know 
the value they are getting for their tax dollars and the difference that 
government departments are making in their lives. 


5.10 | Good performance reporting is also fundamental to effective 
governance and accountability to Parliament. In order to monitor * 
government programs and services effectively, parliamentarians need to be 
provided with timely, accurate information about the cost and performance of 
key government programs. They also need reports that are relatively short 
and easy to read. 


5.11 Departments should also use the same kind of performance 
information to help them produce better results. As we said in our 

2000 Report, Chapter 20, Managing Departments for Results and Managing 
Horizontal Issues for Results, we associate good performance reporting with 
sound performance management. Good performance reports intended for 
external accountability should reflect how departments are managed 
internally. In our view, they should be a routine spin-off from sound 
performance management in a department. 


5.12 For some time, the government has sought to improve the quality of — 
information provided to parliamentarians by government departments. 

In 1981, it created the Estimates Part HI in an effort to improve and expand 
the information on planned departmental spending and on departmental 
performance. In 1995, Parliament initiated the Improved Reporting to 
Parliament Project, which separated the planning and reporting documents in 
an attempt to provide parliamentarians with an improved structure for 
delivering complex information. 


5.13 About 90 federal departments and agencies now submit a performance 
report to Parliament every fall. Each report should outline the 
accomplishments of the department to the end of the fiscal year just 
completed against the commitments made in its report on plans and priorities 
published the previous year. 


5.14. Through our work we have supported parliamentarians in their efforts 
to improve the quality of departmental reporting (Exhibit 5.1). In 1997 we 
examined the state of the government’s reporting regime. We found that the 
reporting framework was basic but sound and that it provided a promising 
start for reporting. However, when we looked again at performance reporting 
in 2000, we were disappointed that only modest improvement had been made 
in improving the quality of departmental performance reports. 


5.15 The Eighth Report of the Standing Committee on Public Accounts 
for 2001 emphasized the importance of good-quality performance reporting. 
The Committee recommended that we “conduct random audits of the 
information contained in the performance reports of departments and 
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agencies in order to verify, among other things, that the information 
contained in these reports is a fair representation of accomplishments against 
goals and objectives.” 


5.16 In 2002 we responded to this request by producing a formal set of 
criteria for good-quality performance reporting and presented the criteria as a 
model for rating departmental performance reports. These criteria elaborated 
on those developed in our earlier work. 


5.17. Among other initiatives to advance performance reporting, each year 
the Treasury Board Secretariat produces guidelines for departments to use 
when they prepare their performance reports (Exhibit 5.2). The guidelines 
include performance reporting principles that are generally consistent with 
our Criteria. 


Exhibit 5.1 Our previous audits and studies 


1988 


1988 Report of the Auditor General, Chapter 5, Information for Parliament—Audit of the Estimates Documents 


We noted that Part Ill had steadily improved since 1981 and represented the best single source of information on 
departmental programs. However, we concluded that it did not yet provide a fully satisfactory basis for 
accountability. 


992 


1992 Report of the Auditor General, Chapter 6, Information for Parliament—Departmental Reporting 


We observed that departmental reporting did not provide the breadth of information needed. We highlighted 
inconsistencies between the way departments carried out their business and what they reported. In particular, we 
noted a weakness in reporting results, performance, and effectiveness. 


ISIS 


1997 Report of the Auditor General, Chapter 5, Reporting Performance in the Expenditure Management System 


We concluded that progress had been made in reporting departmental performance expectations and 
accomplishments and that these efforts needed to be given time to mature. Nevertheless, progress had been 
sufficient to allow us to find examples of good practices that, collectively, demonstrate that the key elements of 
adequate reporting to Parliament can be provided. 


2000 


2000 Report of the Auditor General, Chapter 19, Reporting Performance to Parliament: Progress Too Slow 


We recognized that moving to a results-based culture is not easy and takes time. However, we expected to find 
noticeable progress and were disappointed that only modest improvement had been made. At the present pace, it 
would take too many years for good reporting to become routine. 


2002 


April 2002 Report of the Auditor General, Chapter 6, A Model for Rating Departmental Performance Reports 


We presented our model for rating departmental performance reports and hoped that departments and agencies 
would use it to improve their performance reports. 


2003 


2003 Status Report of the Auditor General, Chapter 1, Rating Departmental Performance Reports 


| We used our rating model to rate nine departmental performance reports. We found that progress in improving 


performance reporting remains disappointing, and that the importance of reporting on horizontal issues was not 


well recognized. 
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Criteria for rating departmental performance reports 
5.18 Our model for good-quality departmental performance reports 
encompasses the following five criteria: 

* Organizational context and strategic outcomes are clear. 

+ Performance expectations are clear and concrete. 

* Key results are reported against expectations. 

* Performance information is credible and balanced. 


* Use of performance information is demonstrated. 


Exhibit 5.2 Key government performance reporting initiatives 


1981 The government committed itself to provide Parliament with improved and expanded information in the Estimates. 
In particular Part III of the Estimates was designed to provide information to Parliament on departmental spending 
intentions and about performance and results produced by expenditures previously authorized. 

TISS The government agreed to include summaries of program evaluations in Part III. 

SINS The government revised the Expenditure Management System. As part of this initiative, it launched the Improved 
Reporting to Parliament Project, which split Part Ill of the Estimates into two documents: 

¢ Report on Plans and Priorities—tabled in the spring, it sets targets and the general direction; 

¢ Performance Report—tabled in the fall, it indicates the results achieved against those planned. 
Six departments piloted the new approach. 
The President of the Treasury Board tabled the first government-wide report describing progress made by 
implementing results-based management in federal departments and agencies. The report is part of the fall 
performance package and is tabled in Parliament with the departmental performance reports. 

1996 Sixteen departments piloted the Improved Reporting to Parliament Project. The Treasury Board President tabled 
their performance reports in the House of Commons. 

L997 On 24 April 1997, the House of Commons passed a motion dividing what was known as the Part III of the 
Estimates document for each department or agency into two documents, a Report on Plans & Priorities and a 
Departmental Performance Report. It also required all departments and agencies to table these reports on a pilot 
basis. 

1998 Most departments and agencies submitted reports on plans and priorities and performance reports. 

2000 The Treasury Board Secretariat published Results for Canadians, which emphasized the importance of ensuring 
timely and accurate reporting to Parliament. 

2001 The Treasury Board Secretariat introduced the Results-Based Management Lexicon. This lexicon provided new, 
standardized terminology for results management and reporting. 

The Treasury Board Secretariat published its renewed guidance to departments for the preparation of performance 
reports and introduced six principles for effective reporting. 

2004 The Treasury Board Secretariat replaced the Planning, Reporting, and Accountability Structure Policy with the 


Management, Resources and Results Structure (MRRS) Policy, effective 1 April 2005. 


The policy requires that departments have clearly defined and measurable strategic outcomes, an articulated 
Program Activity Architecture (an inventory of all the programs and activities undertaken by a department or 
agency that are depicted in their logical relationship to each other and to the strategic outcomes to which they 
contribute), and a description of the current governance structure. 

The Treasury Board Secretariat published an integrated Guide for the Preparation of the 2005-2006 Part III of the 


Estimates: Reports on Plans and Priorities and Departmental Performance Reports. The goal of the integrated 
guidelines was to reinforce the complementary features of the two documents and their parallel reporting 


requirements. 
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5.19 A report that fully meets all five criteria would demonstrate the 
attributes of excellent reporting. However, the rating model is not designed to 
provide assurance that information in a performance report is reliable because 
it does not include an audit of performance information. 


5.20. Overall, the five criteria, each of which includes several subcriteria, 
represent reasonable expectations for a credible performance story about the 
difference that a department’s programs and services are making to 
Canadians. The first criterion identifies the business the department is in, the 
second brings forward the commitments made the previous year, and the 
third presents the accomplishments against those commitments. The other 
two criteria provide the basis for assessing whether the results reported are 
believable and how the performance information can be used to produce 
better results in subsequent years. The model is summarized in Exhibit 1.1 of 
our May 2003 Chapter, Rating Departmental Performance Reports. 


5.21. Our model of what makes a good-quality performance report provides 
for five levels of achievement—basic, fair, good, very good, and excellent. We 
anticipate that, as departments learn about what contributes to excellence in 
reporting and what level they have achieved, they will strive to reach a higher 
level for each criterion. In this sense, our rating model is also a learning or 
developmental model. 


5.22 However, because some criteria build on other criteria, it may be 
difficult to achieve a higher level on a later criterion if the level of reporting 
on an earlier criterion is weak. For example, if performance expectations are 
not clearly and concretely stated, it will be difficult to report any results 
against them. 


5.23 When a departmental performance report approaches the upper levels 
on all five criteria, then an auditor might be able to provide written assurance 
to parliamentarians and Canadians that the report fairly presents the 
department’s accomplishments against its expected results. However, because 
auditors normally provide assurance on both the fairness and the reliability of 
information in reports, then additional audit work would have to be done on 
the reliability of the performance information prior to providing this 
assurance. 


Focus of the audit 


5.24 The objective of our audit was to assess departmental performance 
reporting by assessing the overall quality of departmental performance reports 
of a small group of departments to determine the extent of progress made in 
improving the quality of the reports. Specifically, we assessed the progress 
made over a two-year period, between 2002-03 and 2003-04, compared with 
the progress made over nearly a decade, between 1995-96 and 2003-04. 


5.25 We selected the performance reports of 3 departments—Environment 
Canada, Fisheries and Oceans Canada, and Natural Resources Canada. 
These departments were among the 16 pilot departments participating in the 
Improved Reporting to Parliament Project that started in 1995. 
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5.26 We chose these departments because they share some common 
elements. First, many of their programs are oriented toward protecting the 
environment while also promoting resource utilization. Second, much of their 
work involves scientific and other activities that do not provide services 
directly to Canadians but instead contribute to the work of other 
departments, organizations, or levels of government that have a more direct 
impact on Canadians. 


5.27 In using the rating model developed in 2002, we wanted to have 
consistent standards for all of our ratings; thus, we applied current 
expectations for good performance reporting to the reports from all three 


years (1995-96, 2002-03, and 2003-04). 


5.28 We did not audit the role of the Treasury Board Secretariat because the 
President of the Treasury Board tables the performance reports in Parliament 
on behalf of departments and agencies. The primary accountability for the 
performance reports is with departments and agencies. 


5.29 More details on the audit’s objective, scope, and approach are provided 
in About the Audit at the end of the chapter. 


Observations 


Progress continues to be disappointing 


5.30 We expected to find gradual improvements in the quality of 
performance reporting from one year to the next, with a clear increase in the 
quality of the reports over the longer term. However, we found the results to 
be disappointing. 


5.31 As expected, in the three departments’ first efforts as participants in 
the Improved Reporting to Parliament Project, their reports for 1995-96 
achieved ratings that ranged from almost the basic level through to levels 
between fair and good. Our ratings on individual criteria for both the 2002— 
03 and 2003-04 reports ranged from almost the basic level to between good 
and very good. However, only one report among the nine reports rated 
achieved a very good rating on one of the criteria. 


5.32. Each of the three departments introduced some promising practices 
over the past nine reporting years. For example, all three reports of Fisheries 
and Oceans Canada provide a table that shows a crosswalk of the 
Department’s business lines by the strategic outcomes; this enables readers to 
see which parts of the organization contribute to each outcome and who is 
responsible. In both its 2002-03 and 2003-04 reports, Environment Canada 
offers an opportunity for readers to provide feedback through a questionnaire. 


5.33 All three departments told us they made continuous efforts to improve 
their reporting practices from one year to the next. They also said they 
worked to satisfy the guidelines, released by the Treasury Board Secretariat 
each year, for preparing departmental performance reports. For example, in 
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response to Treasury Board Secretariat guidelines, the 2003-04 Performance 
Report of Natural Resources Canada provides a summary of its performance 
that includes key commitments by each strategic outcome together with a 
self-assessment of how well the Department performed and a reference to the 
section in the report where supporting information can be found. 


5.34 In response to the recommendations outlined in the Sixth Report of 
the Standing Committee on Government Operations and Estimates in 2003, 
departments were asked to include a short summary of parliamentary 
committee reports relevant to their work along with a link or reference to 
further information. All three departments did this in their 2003-04 reports. 
This may contribute to a better-informed dialogue between departments 
and House committees. 


5.35 We found that, while parliamentarians would likely be able to readily 
grasp a department’s main business when they read the reports, they would be 
less likely to clearly see the results of the department’s business. For example, 
through the use of Web links, Fisheries and Oceans Canada makes 
information available about the state of the fisheries but does not present this 
information in its performance report. While Natural Resources Canada 
reports periodically to Parliament on the reserve levels of Canada’s most 
important metals, its performance report only states that the reserve levels of 
these metals is a concern. 


5.36 To some extent, our ratings of the reports of the three departments 
reflect the complex nature of their business. As noted earlier, the work of 
these departments may involve the sometimes competing demands of 
resource utilization and resource protection. Further, much of their business is 
science-based, and their services to Canadians may be delivered through 
other organizations. However, while their business may not conveniently lend 
itself to performance reporting, the three departments told us of their ongoing 
efforts to improve the quality of their reporting over the past nine years. 


5.37 In our assessments of the three departments’ performance reports 
against our rating model, we found that the quality of the reports improved 
marginally, at most, over the two-year period. Over the nine-year period, we 
found that two departments achieved some modest improvements while the 
other department showed mixed results. Although we cannot generalize from 
the results of rating three reports in each of three years to trends across all 
government departments, our findings from this audit continue to leave us as 
concerned about the overall quality of reporting as we were in our previous 
audits and studies. 


5.38 Despite examples of promising practices that demonstrate it is possible 
for departments to improve on some elements of reporting to Parliament, 
these promising practices are overshadowed by the disappointing results 
found when we applied our rating model to the reports. These departments 
continue to face many challenges in their performance reporting. 


8 Chapter 5 


Report of the Auditor General of Canada—April 2005 


Organizational context 


RATING SELECTED DEPARTMENTAL PERFORMANCE REPORTS 


5.39 Based on our findings, it is reasonable to suggest that, without greater 
effort by departments and scrutiny by parliamentary committees, any 
significant improvements in the quality of performance reporting may take 
decades rather than just a few years. In our view, this rate of improvement is 
not good enough for parliamentarians and Canadians to be able to hold 
departments and agencies to account for their performance. 


5.40 Our first criterion for good performance reports requires that a 
department clearly communicate what business it is in and what it does for 
Canadians. It can do this by ensuring that its organizational context and 
strategic outcomes are clear and that they reflect the departmental mandate 
and mission. The criterion expects that reports will clearly communicate how 
a department's activities are organized, such as through business lines. It also 
requires a logical sequence from the department’s business lines to the key 
results, which are clearly expressed as planned strategic outcomes. These 
should take into consideration the department’s operating environment. As 
well, this criterion requires key external partners to be identified and a 
credible description of the risks the department faces to be provided. 


5.41 We expected that each report would provide a meaningful performance 
story of the department’s work by placing it in this broader context. We also 

expected that the quality of these performance stories and the descriptions of 
their operating environment would have improved over the nine-year period. 


A good overview of the organizational context is generally provided 


5.42 The reports generally achieved higher ratings on this criterion than the 
other four criteria in our rating model. Overall, for all the reports we rated, 
reporting for this criterion ranged from almost basic to between good and very 
good, with one report achieving a very good rating (Exhibit 5.3). Over the 
two-year period, the quality of reporting remained consistent for two 
departments but improved for the third. Over the nine-year period, two of 
the departments improved on this criterion, while the third department 
declined. Notably, Environment Canada’s reports improved over both 
periods. 


Exhibit 5.3 Organizational context and strategic outcomes are clear—assessment summary 


Fisheries and Oceans Canada 


Natural Resources Canada 


Environment Canada 


0 1 2 3 4 5 
Basic Fair Good Very good Excellent 
Levels 
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Performance expectations 


Some departments have adopted promising practices 
§.43. In both 2002-03 and 2003-04, the reports of Fisheries and Oceans 


Canada and Environment Canada provided good descriptions of their 
operating environments. They also provided accountability structures and 
identified key partners. Both departments provided Web links in their reports 
so that readers can get more information about their partners. This approach 
helps to keep the reports focussed on performance. Fisheries and Oceans 
Canada also reported on the risks that influence the Department in achieving 
its strategic outcomes. 


Some challenges remain 


5.44 A performance report should state the department’s main business by 
providing its mission and mandate. This information tells readers what the 
organization’s purpose is and provides reference to the legislation that makes 
it possible for the organization to do its work. In both 2002—03 and 2003-04, 
the performance reports of Natural Resources Canada did not provide the 
Department’s mission and mandate, while the Department’s report on plans 
and priorities for the corresponding years provided the mandate. The rating 
for these performance reports could easily be improved against our model by 
including both the Department’s mission and mandate. 


5.45 The reports of all three departments could be improved by providing a 
discussion on how their strategic outcomes contribute to relevant 
government priorities. As well, they should discuss any challenges faced in 
balancing the departments’ own work with their contributions to joint work 
with other departments. 


5.46 Our second criterion expects performance expectations to be clear and 
concrete. More precisely, departments should clearly state their commitments 
to Canadians. These commitments or performance expectations are initially 
provided in a department’s report on plans and priorities and should be 
clearly identified and aligned with federal government priorities. The 
expectations are then carried over into the ensuing performance report. As 
well, performance expectations should be expressed as outputs or outcomes. 
They should also specify the direction of planned changes—for example, 
whether certain outputs or outcomes will be increased, maintained, or 
decreased as a result of the department’s program activities. Finally, the time 
frame in which those changes will be made should be specified. 


5.47 We expected departments to state their expectations in a clear and 
precise way, accompanied by sufficient information to clearly show how the 
departments’ activities and strategies would meet these expectations. We also 
expected the quality of these statements to have improved over the nine-year 
period by increasingly being expressed as outputs and outcomes, rather than 
just as activities. 
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Performance expectations are not always clear and concrete 


5.48 Overall, for all the reports we rated, reporting for this criterion ranged 
from basic to fair on our model (Exhibit 5.4). Over the two-year period, two 
of the departments achieved a higher rating in 2003-04 than in 2002-03. 
Over the nine-year period, one department improved, and two showed no 
change. 


Exhibit 5.4 Performance expectations are clear and concrete—assessment summary 


Fisheries and Oceans Canada 


Natural Resources Canada 


Environment Canada 


0 1 2 3 4 5 
Basic Fair Good Very good Excellent 
Levels 


Use of results chains improves reporting 
5.49 In 2002-03 and 2003-04, Fisheries and Oceans Canada’s reports 


presented good results chains or logic models that summarize at a glance the 
Department’s performance expectations, partnerships, business lines, and 
activities. Environment Canada’s 2003—04 Performance Report provided a 
logic model for each business line that also outlines the Department’s 
performance expectations in terms of immediate and intermediate outcomes. 


Improvements are needed 


5.50 Despite being rated from basic to fair on our rating model, some key 
performance expectations presented in the reports of all three departments 
were inconsistent with the expectations presented in their planning reports. 
These changes to the performance expectations were not explained in the 
performance report. 


5.51 The reports could be significantly improved if performance 
expectations were expressed as outputs and outcomes, with each 
commitment having a clearly-stated direction, amount, and time frame. We 
would also like to see a description of the strategies to be used for producing 
better results in subsequent years. 


5.52. The third criterion of our model requires key results to be reported 
against performance expectations. This criterion expects results to be 
presented as outputs or outcomes. It also expects reports to state whether the 
department achieved its commitments and to explain any gaps in 
performance. This includes aligning outputs and outcomes with performance 
expectations, addressing the challenges of achieving expected results, 
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identifying the partners that contributed to the performance outcomes, 
determining the level of resources required, and interpreting the results. We 
believe that this criterion is at the very core of good performance reporting. 


Outputs and outcomes are not widely reported 


5.53 Overall, for all the reports we rated, reporting for this criterion ranged 
from basic to fair, with only one report approaching a good level of reporting 
(Exhibit 5.5). The results reported tended to focus largely on activities rather 
than on the outputs produced and on the outcomes to which they contribute. 
According to our rating model, over the two-year period two departments 
marginally improved their reporting, and the same two departments improved 
by up to one level over the nine-year period. Given the government’s 
emphasis on results-based management over the past decade, we expected to 
see greater improvement on this criterion. 


Exhibit 5.5 Key results are reported against expectations—assessment summary 


Fisheries and Oceans Canada 


Natural Resources Canada 


Environment Canada 


0 1 2 3 4 a 
Basic Fair Good Very good Excellent 


Levels 


Some promising practices are provided 


5.54 Despite finding some limited improvements on this criterion, some 
promising practices are provided. In 2003-04, while all three departments’ 
reports made some reference to evaluations and internal audits, only Natural 
Resources Canada integrated the findings of an evaluation with its reporting 
of departmental results. All three departments also provided an estimate of 
the level of resources used to achieve each strategic outcome. 


§.55 The 2002-03 report from Environment Canada provided an Internet 
link to a table that displayed the Department’s expectations and how well it 
has performed against them. However, this table would be more useful if it 
were shorter and included in the performance report itself. 


Challenges remain 


§.56 The performance reports could be improved by systematically 
integrating the findings from evaluations and audits in discussions of the 
results. Estimates of the resources used to achieve concrete targets should 
also be provided. The gap between the planned results and the actual results 
should be explained. As well, information should be provided so that results 
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achieved can be interpreted, such as trends over the past five years or 
comparisons with other departments. 


5.57 Our fourth criterion requires performance information to be credible 
and balanced. First, it focusses on the quality of performance information and 
the reliability of information sources as the basis for judging the credibility of 
the data. Second, it focusses on whether the reporting of good results is 
balanced with the reporting of shortcomings and whether the level of detail 
for key results is appropriate. This criterion is fundamental to a compelling 
and credible performance report. 


5.58 We expected to find evidence to show that the information was 
credible. We also expected that results falling short of expectations would be 
reported along with the success stories. Further, we expected that the amount 
of information provided for the key results would be proportional to their 
importance. 


Departments do not generally report credible and balanced results 


5.59 Overall, for all the reports we rated, reporting for this criterion ranged 
from almost basic to fair (Exhibit 5.6). Over the two-year period, one 
department improved on this criterion, one showed no change, and one 
declined. There was no improvement on this criterion over the nine-year 
period for any of the departments, and we could not find any promising 
practices. 


Exhibit 5.6 Performance information is credible and balanced—assessment summary 
Fisheries and Oceans Canada 
Natural Resources Canada 


Environment Canada 


0 1 2 3 4 5 
Basic Fair Good Very good Excellent 


Levels 


Many challenges remain 


5.60 The reports could be improved by reporting data sources and 
limitations. They should also provide a discussion on how to interpret their 
performance information, including the relevance or importance of the 
performance achieved. It is important to report performance shortcomings as 
well as successes; this would reduce the risk that users, such as 
parliamentarians, might discount the positive results as biased, incomplete, 


and lacking in credibility. 


Report of the Auditor General of Canada—April 2005 


Chapter 5. | 13 


RATING SELECTED DEPARTMENTAL PERFORMANCE REPORTS 


Use of performance information 


§.61 Managing well to improve results is a challenge. The previous four 
criteria are based on a more traditional approach to assessing performance 
reporting. For these criteria, the organizational context must be provided; the 
performance should be defined, measured, and reported against expectations; 
and the information must be credible and balanced. However, even if all 
these criteria have been met, if feedback on past performance is not used to 
improve future results, then the use of the report may be restricted to 
addressing the external accountability requirement of reporting to 
Parliament. 


5.62 Performance information is not collected just to create accountability 
reports; it should also be used to help departments make sound internal 
decisions to manage for better results. For example, when evaluations of 
programs provide evidence about which programs work well and which ones 
do not, then funding could be redirected away from programs that have been 
shown not to work well to those that do work well. In addition, the 
performance report should provide a credible discussion about the capacity of 
a department to produce sustainable results by continuing to perform well in 
the future. A good-quality report also highlights lessons learned and identifies 
how weak performance will be corrected in future years. 


5.63 We expected that performance reports would demonstrate ways in 
which departmental management is using performance information, 
including how it will use the information to establish future performance 
expectations and improve future performance. 


Performance information is seldom used 

5.64 Overall, for all of the reports we rated, reporting for this criterion 
ranged from almost basic to between basic and fair, with most reports 
achieving only a basic rating (Exhibit 5.7). According to our rating model, 
over the two-year period one department improved, one showed no.change, 


and one declined. Over the nine-year period, two departments improved on 
this criterion, while the report of the third department remained the same. 


Exhibit 5.7 Use of performance information is demonstrated—assessment summary 


Fisheries and Oceans Canada 


Natural Resources Canada 


Environment Canada 


Levels 0 1 2 3 4 5 
Basic Fair Good Very good —_ Excellent 
Levels 
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Use of performance information in decision making improves reports 


565 In 2003-04, Natural Resources Canada reported how it used the results 
provided by an evaluation about road surfaces and the fuel efficiency of 
vehicles. Based on this information, resources previously devoted to research 
on concrete roads were redirected to research on other road materials. We 
consider this a promising practice because it illustrates how the Department 
used performance information to potentially improve future performance. 


Future performance needs to be discussed 


5.66 ‘The reports could be improved by including both explanations of 
corrective action taken to address performance issues and plans to improve 
performance. Reports should also discuss the risks associated with continuing 
to either meet or make progress on performance targets in the future. The 
ratings on this criterion suggest that this could be one of the most challenging 
aspects of performance reporting. In facing this challenge, departments will 
need to incorporate good performance information into their 
decision-making processes, evaluate whether they are heading in the right 
direction, and use the information to correct problems. 


Conclusion 


5.67 This is the second time that we have used our model to rate a selection 
of performance reports. However, it is the first time we have compared a 
department’s reports over one or more periods to assess progress in improving 
the quality of reporting. Overall, we found that progress in improving 
performance reporting to Parliament continues to be a challenge for the three 
departments, even after almost a decade of effort. 


5.68 We found that, between 1995—96 and 2003-04, two departments at 
least doubled the number of pages in their performance reports. We are not 
convinced that the greater length has translated into a proportional increase 
in the quality of the reports. Some of the greater length can be attributed to 
additional requirements from the Treasury Board Secretariat and the trend 
toward greater complexity of the information. However, some of the 
increased length is a reflection of more information being reported than is 
necessary and information being too detailed for the intended readers. While 
the 1995-96 reports were handicapped by focussing largely on activities 
rather than on outputs and outcomes, their performance stories were more 
succinct and, therefore, more understandable. 


5.69 The findings from this audit are consistent with our observations from 
our previous work. In 1988, we concluded that reports to Parliament did not 
provide a fully satisfactory basis for accountability. In 1992, we again 
indicated that these reports did not provide the necessary breadth of 
information. In 1997, when the government reviewed its Expenditure 
Management System, we stated that progress in performance reporting to 
Parliament was being made. In our 2000 Report, Chapter 19, Reporting 
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Performance to Parliament: Progress Too Slow, we concluded, “At the current 
rate of progress, it will be many years before good performance reporting 
becomes routine.” In 2003, when we rated nine reports, we found that 
progress in performance reporting remains disappointing. 


5.70 In our previous work, we listed five factors that contribute to the 
quality of performance reporting. We believe these are still valid. 


5.71 First, the basic principles of good reporting are frequently not well 
understood or applied by departments. This remains a factor, despite the 
Treasury Board Secretariat’s annual guidelines that reinforce the principles of 
good reporting. 


§.72 Second, performance reporting takes place in a political environment. 
This continues to be a factor because the value of reporting shortcomings has 
not yet been widely accepted. Balanced reporting, that is, admitting to 
shortcomings as well as successes, is apparently not yet part of the 
management culture of government. 


5.73 Third, there are no incentives for good reporting practices or sanctions 
applied for poor reporting. Because House committees do not control 
departmental budgets, as their counterparts do in some other jurisdictions, it 
is more difficult for parliamentarians to encourage good reporting or penalize 
poor reporting through the use of financial measures. As well, some 
jurisdictions scrutinize performance reports and name the poorly-performing 
departments as a way of motivating them to improve their reports. 


5.74 Fourth, many departments do not consider performance reports to be a 
high priority. Often, the reports do not get the involvement or attention of 
senior departmental management that they should. This may be because 
information in performance reports is neither used by departments to manage 
for results nor used by the government to manage government-wide 
initiatives. 


5.75 Fifth, despite initiatives of Parliament and the government aimed at 
improving the quality of performance reports over the past few decades, in 
our view parliamentary committees have not taken advantage of the reports 
in their discussions with the departments audited. Greater interest in and 
scrutiny of the reports by parliamentary committees would encourage 
departments to improve the quality of their performance reports. 


5.76 These factors cannot be overcome easily. They can only be addressed 
by fundamentally changing the management culture of government. 
Consequently, we are not making any formal recommendations to 
departments because our message continues to be that they should make a 
greater effort to better understand the basic principles of good reporting and 
they should continue to work on improving their performance reports. While 
the onus is on departments to improve these reports, the Treasury Board 
Secretariat should also continue to help departments by reviewing their 
performance reports and providing leadership and opportunities to increase 
their understanding and acceptance of the principles and practices of good 
performance reporting. 
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5.17 Parliamentary review of performance reports through the various 
standing committees is essential for improving the quality of performance 
reports. Committees could demand clarification and explanations of 
departmental results. They could then challenge departments and agencies 
on how well they have performed against their commitments to Canadians. 
Only when parliamentarians are engaged with departments in an ongoing 
dialogue about departmental performance will the quality of these reports 
improve. 


Government’s response. We welcome the efforts of the Auditor General in 
assessing departmental performance reports and in providing advice on areas 
that need improvement. The Government of Canada remains strongly 
committed to ongoing improvements in reporting to Parliament, recognizing 
that sound, transparent, and effective reporting is key to supporting 
Parliament in its role of holding the government to account for the 
management and use of public funds. 


Reports on plans and priorities and departmental performance reports are 
primary instruments of accountability to Parliament. They are a mechanism 
through which ministers and their organizations demonstrate responsibility 
and accountability. These reports explain to Parliament their departmental 
plans, priorities, and expected results, and account for the performance 
achieved. In collaboration with departments, the Treasury Board Secretariat 
(TBS) has recently revised its departmental guidelines and reinforced the 
results-based focus of these reporting instruments. The TBS will continue to ° 
work with departments, parliamentarians, and the Auditor General to help 
improve practices highlighted in the Report. 


In December 2004, the Treasury Board Secretariat introduced the 
Management, Resources and Results Structure (MRRS) Policy. It will provide 
a standard, government-wide approach to planning and reporting on resource 
expenditures and results. To ensure better quality of performance reporting, 
the TBS will provide assistance and guidance to departments as they develop 
and adjust their management, resources, and results structures. This is a 
major shift in planning and performance reporting practices and will require a 
phased-in approach to full implementation. We expect that the MRRS policy 
will serve as a consistent and enduring foundation for financial and 
non-financial reporting to Parliament. 


In the coming months, the government, in collaboration with 
parliamentarians, will develop a blueprint for improved reporting to 
Parliament that will better respond to needs of parliamentarians and other 
stakeholders for transparent, timely, and easy-to-understand performance 
information. The government will also invest in technological systems that 
support performance data and reporting. Combined, these efforts should 
bring about a gradual shift in the management culture and in performance 
reporting practices across government departments. 


We welcome the opportunities for continued collaboration with the Auditor 
General, parliamentarians, departments, and others to enable the required 


shift toward improved performance reporting and management practices. 
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About the Audit 


Objective 


The objective of this audit was to assess departmental performance reporting by assessing the overall quality of 
departmental performance reports of a small group of departments to determine the extent of progress made in 
improving the quality of the reports. Specifically, we assessed the progress made over a two-year period, 
between 2002-03 and 2003-04, compared with the progress made over nearly a decade, between 1995—96 
and 2003-04. 


Scope 


We selected the performance reports of three departments—Environment Canada, Fisheries and Oceans Canada, 
and Natural Resources Canada. These departments were among the second round of pilot departments participating 
on the Improved Reporting to Parliament Project, which provided for the splitting of the Estimates Part III into 
planning and reporting documents. 


We did not audit the Treasury Board Secretariat. 


Approach 


We assessed three performance reports from each of three departments against our rating model. These reports 
covered the period ending 31 March 1996, 31 March 2003, and 31 March 2004. We followed the approach that we 
described in our April 2002 Report, Chapter 6, A Model for Rating Departmental Performance Reports, and applied 
to nine departmental performance reports in our May 2003 Report, Chapter 1, Rating Departmental Performance 
Reports. 


In the rating model, each of the five criteria is represented by a continuum that has five levels or stages of 
development. These are basic, fair, good, very good, and excellent. An exceptional performance report would 
achieve an excellent rating in each criterion. 


Three raters (two from separate teams within our Office and another from outside the Office) applied the rating 
model to the performance reports to produce their rating. They then discussed their ratings to arrive at a consensus 
rating. 


Although we used the rating model developed in 2002, we applied current expectations for good performance 


reporting to the reports from all three years (1995—96, 2002-03, and 2003-04). 


We did not audit the departments’ systems and procedures for producing the information included in their reports or 
reach a conclusion on the reliability of the performance information contained in the reports. 


Audit team 


Assistant Auditor General: Hugh McRoberts 


Principal: Barry Leighton 


Doreen Deveen 
John McGrath 
Anupheap Ngoun 
Ruth Sullivan 


For information, please contact Communications at (613) 995-3708 or 1-888-761-5953 (toll-free). 
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Chapter 


Indian and Northern Affairs Canada 


Development of Non-Renewable 
Resources in the Northwest Territories 


All of the audit work in this chapter was conducted in accordance with the standards for assurance engagements set by the 
Canadian Institute of Chartered Accountants. While the Office adopts these standards as the minimum requirement for our audits, 
we also draw upon the standards and practices of other disciplines. 


Table of Contents 


Main Points 


Introduction 


Historical overview of non-renewable resources in the North 
An evolving regulatory framework for developing resources 
Parties involved in resource development 

Process for approving resource development applications 
Focus of the audit 


Observations and Recommendations 


Governance of resource development 


A reduced operational role for the Department in regulating development 
The Department needs to take a more active role to fulfill its responsibilities 
Guidance on key terms in the legislation needs to be provided 

Regulations for water should be established 


The Department needs to establish an effective process to ensure that the boards 
have the appropriate resources 


Renewing the Department’s role 


The Department needs to hold the boards accountable for managing the process 
Good reporting begins with a clear understanding of the accountability relationship 
The Department needs to establish an effective working relationship with the boards 


Conclusion 


About the Audit 
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Indian and Northern Affairs Canada 


Development of Non-Renewable 
Resources in the Northwest Territories 


Main Points 


6.1 | Non-renewable resources offer enormous potential for economic 
development in the Northwest Territories (NWT). Yet the investment 
climate for this development is uncertain, in part because Indian and 
Northern Affairs Canada has not adequately managed its role in the process 
that considers development projects. 


6.2 This includes not providing guidance on some of the ambiguous terms 
in the governing legislation or on establishing water standards permitted by 
legislation. It also includes not requiring boards to be accountable for 
managing their role in the process without impinging on the decisions they 
take as quasi-judicial bodies. 


Background and other observations 


6.3 Our audit examined how well Indian and Northern Affairs Canada 
(INAC) has managed its responsibilities that form part of the application and 
licensing process for the development of non-renewable resources in the 
Northwest Territories. 


64 With the signing of land claim agreements in the Northwest Territories 
and the passage of the Mackenzie Valley Resource Management Act (MVRMA), 
the federal government created a series of boards to regulate the use of land 
and water and protect the environment. When these boards were created, the 
Department felt that the federal government's best course of action was to 
leave the boards to administer the process on their own in order to ensure that 
the Aboriginal peoples of the NWT understood that the federal government 
was sincere in relinquishing control. That decision seven years ago has 
contributed to a regulatory environment that, today, needs strengthening. 


65 As the federal government continues negotiating the transfer of 
responsibilities to the Northwest Territories, it is important for Canadians 
that the process for developing non-renewable resources that the NWT may 


inherit is well managed. 


66 Furthermore, seven years since the passage of the Act, it is timely for 
the Department to re-examine its approach to managing the process and 
strengthen it in the areas described in this report to ensure that it is 
sufficiently robust to meet the challenges and realize the opportunities of the 
coming decade. 

The Department has responded. Indian and Northern Affairs Canada 


agrees with all the recommendations and has committed to taking action to 


address the concerns we raise in this chapter. 
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INDIAN AND NORTHERN AFFAIRS CANADA—DEVELOPMENT OF NON-RENEWABLE RESOURCES IN THE NORTHWEST TERRITORIES 


Introduction 


Historical overview of non-renewable resources in the North 


6.7 The development of non-renewable resources is vital to the economic 
development of Canada’s North. In the Northwest Territories (NWT), this 
means the development of, primarily, minerals and natural gas. 


6.8 | While the discovery of metals goes back to the 18th century, modern 
mining began with the Yukon gold rush in the late 1800s. Oil was later found 
at Norman Wells, NWT, in 1918. Fifty years later, gas was discovered in the 
Mackenzie-Beaufort Delta; however, it has remained inaccessible to North 
American markets because there has been no way to bring it south. Today, 
30 years later, a new application has been made to build a pipeline. 


6.9 | Onthe mining side, in addition to gold, lead and zinc are the two 
metals most commonly extracted in the NWT. But diamonds represent the 
new mining frontier. 


6.10 In 2003, the gross domestic product of the Northwest Territories was 
$3,332 million, about 13 percent higher than the year before. Since 2000, 
about two years after miners extracted the first diamonds, the NWT economy 
has grown annually by about 10 percent. In the same period, the share of the 
economy represented by mining (excluding gas and oil) grew from just under 
24 percent to about 42 percent, and the size of the mining industry more than 
doubled. The industry will get another boost once a third diamond mine is up 
and running, expected in 2006. The other potential impetus to the NWT 
economy is natural gas in the Arctic, which will depend on the construction 
of the proposed Mackenzie Valley pipeline. 


An evolving regulatory framework for developing resources 


6.11 The environmental consequences of resource development have been 
a matter of growing concern in Canada since the late 1960s. With the federal 
government’s commitment in 1974 to review the environmental effects of 
federal decisions throughout Canada, all development projects on federal 
lands or in areas under federal jurisdiction became subject to screening to 
ensure the least possible damage to the environment. In 1984, the 
Environmental Assessment and Review Process Guidelines Order codified 
what had been a largely unwritten process arising from the 1974 Cabinet 


policy. 
6.12. In 1995, the Canadian Environmental Assessment Act (CEAA) replaced 
the Order and became the basis for conducting environmental assessments in 


areas of federal jurisdiction. 


6.13 The devolution of federal responsibilities to the territorial governments 
and the existence of Aboriginal land claim settlements have made 
environmental assessments in the North more complex. On | April 2003, the 
Government of Yukon took over the management of its non-renewable 
resources, except in areas where transboundary development is proposed. 
However, environmental assessment responsibilities will remain under federal 
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authority, but with significant participation from the Yukon First Nations. In 
Nunavut, the land claim settlement established a process for issuing land use 
permits and water licences as well as a process for dealing with environmental 
concerns. In the NWT, except in the Inuvialuit land claim area, 
environmental considerations and the issuing of licences and permits fall 
under the 1998 Mackenzie Valley Resource Management Act (MVRMA) and 
the Sahtu and Gwich’in land claims legislation (Exhibit 6.1). As other claims 
are settled, such as the Tlicho claim, where the ratifying legislation is before 
Parliament, and the Deh Cho claim, which is currently being negotiated, the 
MVRMA will be amended to incorporate them into the process. 


Exhibit 6.1 Aboriginal settlement areas in the Northwest Territories 


[J Settlement regions 


mt 


Nunavut 


Aa 


Wek’eezhii 
(Tlicho Area) 


North Slave 
South Slave 


Alberta 


British Columbia 


Saskatchewan 


6.14 The MVRMA was created to meet a federal obligation under the 
Sahtu and Gwich’in land claim agreements and implementing legislation. It 
called for the creation of public boards to manage the application process for 
the development of renewable and non-renewable resources in the 


Mackenzie Valley. 
Parties involved in resource development 


6.15 The application for non-renewable resources development begins 
when a developer applies toa MVRMA-created board for a land use permit 
Type A and 


Type B; Type A is for larger and/or more complex projects). Once an 


and/or water licence (there are two types of water licence 


application is received, several organizations become involved. 
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6.16. Minister for Indian and Northern Affairs. The MVRMA assigns to 
the Minister several responsibilities for resource management in the 
Mackenzie Valley. These include operational and governance responsibilities. 
At the operational level, responsibilities include 


* adopting, with or without modifications, or rejecting recommendations 
of the Mackenzie Valley Environmental Impact Review Board; and 


* approving Type A water licences issued by the land and water boards. 


6.17. At the governance level, section 82 of the MVRMA sets out the 
principal responsibility of the Minister for Indian and Northern Affairs in 
relation to land and water regulation; it gives the minister the authority, after 
consultation with a board, to give written policy directions binding on that 
board regarding its responsibilities for land and water regulation. Other 
responsibilities include 


* making regulations about water quality, after consultation with the 


Mackenzie Valley Land and Water Board; 


* appointing board members where, except for the chair, half are selected 
from First Nation nominations and the other half from government 
nominations; 


* appointing the board chairs; and 


* specifying the form of the annual report that the boards must submit to 
the Minister. 


6.18 Gwich’in, Sahtu, and Mackenzie Valley land and water boards. 
These federally created boards are responsible for regulating the use of land 
and water and the deposit of waste in the Mackenzie Valley, for the benefit of 


‘NWT residents and all Canadians. The boards receive applications for the 


land permits and/or water licences needed before such projects can proceed. 
If a project is limited to Gwich’in or Sahtu land, the board for the region in 
question manages the application process. If the project crosses boundaries or 
is on land not covered by a settled land claim, the Mackenzie Valley Land and 
Water Board is responsible. 


6.19 Gwich’in and Sahtu land use planning boards. Created by the federal 
government in 1998, these boards are responsible for preparing land use plans 
and overseeing land use in the Gwich’in and Sahtu settlement areas. When a 
developer submits an application for development on Gwich’in or Sahtu land, 
the proposal is forwarded to the planning board for the region in question to 
ensure that the application complies with the land use plan for the area. 


6.20 Parties affected by the development proposal. Before the land and 
water board can proceed with an application, it must notify any organization 
or individual affected by the proposal. This could include any number of 
federal and territorial organizations and local governments, as well as 
organizations associated with the Gwich’in, Sahtu, or Tlicho land claims. 


6.21. Mackenzie Valley Environmental Impact Review Board (MVEIRB). 
The federal government created this board in 1998 to conduct environmental 
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assessments and reviews of resource development applications referred to it 
by other boards and organizations or on its own motion. 


Process for approving resource development applications 


6.22. The steps in the approval or rejection of an application for a permit or 
licence are set out in the Mackenzie Valley Resource Management Act. 


6.23 The boards are key to making the process work. This process 
involves four boards: the Mackenzie Valley Land and Water Board; its two 
panels, the Gwich’in Land and Water Board and the Sahtu Land and Water 
Board; and the Mackenzie Valley Environmental Impact Review Board. 


6.24 Inthe past, Indian and Northern Affairs Canada and the Northwest 
Territories Water Board had primary authority over resource development. 
With the Inuvialuit land claim agreement in 1984, the Sahtu and Gwich’in 
land claim agreements in the early 1990s, and the passage of the MVRMA, 
the creation of a number of boards in the Northwest Territories changed the 
Department’s role by introducing a new way of managing resource 
development in the NWT. The federal government no longer directly 
controlled the issuing of permits and licences for land and water use. Nor was 
it responsible for assessing the potential environmental impacts of 
development. This meant that the First Nations and Inuvialuit obtained what 
they had been seeking for some time: participation in regulating the use of 
resources on their lands, with the boards being the vehicle for that 
participation. 


6.25 The boards have become the filter through which must pass every 
application for resource development that requires a land use permit or water 
licence. There are no exceptions; the Minister cannot bypass the process 
unilaterally or intervene before the process has run its course and reports 
have been issued. 


6.26 Timelines are established. At the beginning of the process, a board 
will formally acknowledge that it has received an application only when it is 
satisfied that the application is complete. This acknowledgement establishes 
the period within which the applicant can expect a response, ranging from 
42 days for a land use permit to up to 90 days for a water licence, unless the 
application is referred to the MVEIRB for an environmental assessment. 


6.27 Applications must conform to land use plans. Land use plans provide 
for the conservation; development; and use of land, water, and resources in 
an area covered by a settled land claim. Once an application is complete, the 
board will check whether it conforms to the land use plan that applies; if it 
does not, and cannot be amended, it is rejected. 


6.28 Applications must go through a preliminary screening. The first 
stage in the review of a complete application is a preliminary screening by the 
appropriate land and water board or any other regulatory authority that has a 
power to issue a license or permit. If any of these bodies determines that the 
proposed project “might have a significant impact on the environment or 
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might be a cause for public concern,” it refers the application to the MVEIRB 
for an environmental assessment. 


6.29 The land and water boards must send the application to various expert 
reviewers for their comments on the likelihood of environmental impact and 
public concern. These reviewers include various departments and agencies of 
both the federal government and the Government of the Northwest 
Territories, as well as the First Nation communities affected. 


6.30 If the proposed development is unlikely to cause a significant public 
concern or adverse environmental impact, either the Gwich’in, Sahtu, or 
Mackenzie Valley land and water boards decides whether the licence or 
permit should be issued and establishes the terms of the licence. 


6.31 The Mackenzie Valley Environmental Impact Review Board 
manages the environmental assessment process. When any one of the 
preliminary screening bodies identified by the MVRMA believes that a 
proposed development outside of a local government boundary might cause a 
significant adverse impact on the environment or might be a cause of public 
concern, the Review Board is required to do an environmental assessment 
even without a referral from a land and water board. If the proposed 
development is inside a local government boundary, the condition for 
conducting an environmental assessment changes from “might” to will 
“likely” cause a significant adverse impact on the environment. The Review 
Board can also decide on its own to conduct an environmental assessment. 


6.32. On completing an environmental assessment, the Review Board will 
determine whether, in its opinion, the project is likely to have a significant 
adverse impact on the environment or to be the cause of significant public 
concern. If the Board finds that a project does not give rise to one of these 
conditions, then it can determine that no further review need be done. If it 
finds that the proposal does meet one of these conditions, it can order a more 
extensive environmental impact review. It can also recommend approval, 
subject to sufficient mitigating conditions, or it can recommend that the 
Minister reject the proposal without any further review. 


6.33 The course of action open to the Minister upon receiving a report from 
the Review Board is limited. He or she can adopt the recommendations, refer 
them back to the Board for further consideration, reject them, or after 
consulting the Board, adopt the recommendation with modifications. The 
Minister cannot modify the recommendations without consulting the Board. 


Focus of the audit 


6.34 Our audit examined how well Indian and Northern Affairs Canada is 
managing its responsibilities for the process set out in the Mackenzie Valley 
Resource Management Act for the development of non-renewable resources in 
the Northwest Territories (apart from the Inuvialuit Settlement Region). We 
looked at the process from the point at which one of the regulatory and 
environmental assessment boards receives an application for a permit and/or 


licence until a decision by one of those boards is made. 
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Governance of resource development 


6.35 We did not audit any of the boards’ responsibilities for their practices, 
procedures, or internal administration; nor did we examine the roles that 
other federal departments and agencies play in the process. However, we did 
interview officials from the boards to understand how the Department is 
managing its responsibilities. 


6.36 Further details on our audit objectives, scope, approach, and criteria 
are presented in About the Audit at the end of the chapter. 


Observations and Recommendations 


A reduced operational role for the Department in regulating development 


6.37 With the signing of land claims, the creation of the boards, and the 
passage of the Mackenzie Valley Resource Management Act (MVRMA), the 
federal government effectively transferred part of its existing responsibilities 


regulating the 
use of land and water and examining the environmental impacts of non- 


for managing the development of non-renewable resources 


renewable development proposals—to several boards in the Northwest 
Territories. Those boards are the Mackenzie Valley Land and Water Board; its 
two panels, the Gwich’in Land and Water Board and the Sahtu Land and 
Water Board; and the Mackenzie Valley Environmental Impact Review 
Board. 


6.38 The Act provides an overall framework for that process that includes a 
series of discrete steps. All the parties we spoke to understood the framework, 
the steps in the process, and the decisions that must be made at each of the 
points. 


6.39 Having transferred this regulatory authority to the boards, the 
Department retained several responsibilities that need to be managed well if 
the process is to work as intended. It maintained those responsibilities 
because it recognized that the investment climate in the Northwest 
Territories (NWT) could be influenced by how well the process worked. 
Accordingly, we looked at how the federal government is managing its 
responsibilities associated with the process for the development of non- 
renewable resources in the NWT. 


6.40 In managing those responsibilities, we expected the Department to 
have provided adequate direction to ensure that the details to make the 
process run smoothly were in place. We also expected the Department to 
have determined that the boards had the required resources, both financial 
and non-financial, to carry out their functions. Finally, we expected the 
Department to have managed its responsibilities and authorities in a way that 
demonstrated that the roles, capacities, and accountabilities of those 
involved in the process are clear and fulfilled. 
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The Department needs to take a more active role to fulfill its responsibilities 


6.41. While Indian and Northern Affairs has transferred some of its 
operational responsibilities for resource development in the Northwest 
‘Territories, it continues its governance role in several areas. Through the 
audit, we identified four areas where the decision the Department took some 
seven years ago regarding its scale of its involvement needs revisiting. These 
include 


* providing guidance on key terms in the legislation, 
* establishing regulations for water quality, 


* ensuring the boards have the necessary resources to carry out their 
functions, and 


* requiring the boards to be accountable not only for financial 
performance but also for the way in which they manage their 
responsibilities for the process. 


6.42 According to Indian and Northern Affairs officials, the Department 
decided to keep its involvement to a minimum given the need to assure 
Aboriginal peoples of the NWT that the federal government was sincere in 
relinquishing control. Seven years later, that decision has resulted in 
ambiguity surrounding the regulation of non-renewable resources, which has 
raised the uncertainty that the process will be applied consistently. It would 
seem timely today for the Department to address the issues we raise to help’ 
ensure that the process is sufficiently robust to meet the challenges and to 
realize the opportunities of the coming decade. 


Guidance on key terms in the legislation needs to be provided 


6.43 Under the Mackenzie Valley Resource Management Act, a land and water 
board or any regulatory authority must conclude at the preliminary screening 
of a proposal whether it “might have a significant impact on the environment 
or might be a cause for public concern.” If the authority believes the proposal 
might have a significant impact, it will refer the application to the Mackenzie 
Valley Environmental Review Board for an environmental assessment. 


6.44 Interested parties we interviewed indicated that before submitting an 
application or registering a matter that could cause public concern, they 
should be entitled to guidance on how key terms such as “might have a 
significant impact on the environment” or “may be a cause for public 
concern” are to be interpreted. In this regard, the Department has given the 
land and water boards no such guidance, nor do applicants for permits receive 
any direction on how the land and water boards might interpret the terms. 
Yet, in other similar legislation such as the Canadian Environmental Assessment 
Act, Environment Canada has issued draft guidelines that are being used for 
assessing the role of public participation in similar processes under its 
jurisdiction. 


6.45 In 2004, the Review Board issued guidance on how the word “might” 
could be applied in practice, and there are several environmental publications 
that provide direction on interpreting the word “significant.” But the boards, 
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or any party in a similar position, are not required to use this guidance or any 
particular interpretation. 


6.46 Consequently, any of the land and water boards or any regulatory 
authority can require the Review Board to conduct an environmental 
assessment without having to be accountable for that decision. 
Environmental assessments, which by their nature take time and increase 
costs, are an important component in the regulatory process and should only 
be used when justified. 


6.47. Recommendation. Indian and Northern Affairs Canada, in 
consultation with the boards under the Mackenzie Valley Resource Management 
Act, should develop guidelines for clarifying key terms in the legislation. 


Department’s response. The Department, with the boards throughout the 
Northwest Territories (NWT), has developed a process known as the NWT 
Board Forum. Through this forum, the Department will work with the boards 
to develop guidelines to clarify key terms of the legislation. These will be 
based on the precedent work already completed through the Canadian 
Environmental Assessment Act. A working draft for external consultation will 


be completed by | April 2006. 


Regulations for water should be established 


6.48 When the land and water. boards issue water licences under the 
authority of the Northwest Territories Waters Act and the MVRMA, they 
require the licensees to meet certain conditions such as measures to mitigate 
the environmental impacts of the use of water or the deposit of waste. 
Applicants for licences or permits should be able to know before they submit 
their proposals the standards for water use and waste disposal that they must 
meet. In that way, they would be able to demonstrate in their project plans 
how they will meet those standards. 


6.49 In fact, the Northwest Territories Waters Act provides for the Minister for 
Indian and Northern Affairs, working with the boards, to make regulations 
governing the quality of water. Similarly, the MVRMA gives the Minister the 
authority to provide written policy directions regarding land and water 
regulations. 


6.50 However, the Department has chosen not to exercise these authorities. 
Consequently, when completing an application for a water licence, applicants 
do not know whether they are to meet an international standard of water 
quality, a national or territorial standard, a standard specific to the 
development site’s environment, or the highest standard established by 
science. 


6.51 This absence of direction on standards for water can raise the risk of 
confusion and uncertainty over the stringency of the requirements that 
applicants are to meet in order to have their applications approved. 


6.52 Recommendation. Indian and Northern Affairs Canada, in 
consultation with the boards under the Mackenzie Valley Resource Management 
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Act, should develop standards for water and the Minister should direct the 
boards to use the standards. 


Department’s response. In consultation with the boards and water users, the 
Department will ascertain the information needs (with respect to water 
standards used by the boards to set licence terms and conditions) of water 
users and the best form to provide proponents with certainty. A report on 
information needs will be completed by the end of 2006. 


In consultation with the boards, the Department will develop water standards 
and set them out in codes, guidelines, policy, or regulations, as best fits the 
need. A completion date will be determined as part of the consultation. 


The Department will improve the system for notification to the boards of 
various standards. This will be an ongoing process. 


The Department needs to establish an effective process to ensure that the boards have 
the appropriate resources 


6.53 The land and water boards are to conduct their business at arm’s length 
from government and the First Nations that nominate them. Their business 
includes deciding on the use of land and water and protecting the 
environment. The business of the Mackenzie Valley Environmental Impact 
Review Board also includes recommending the approval of projects, subject 
to taking measures to mitigate significant adverse environmental impacts. 


6.54 It is important that the boards have the resources to carry out their 
functions, because their decisions and recommendations, once the Minister 
has accepted them, are binding. There is no appeal except to the courts. 
Given the significant consequences of the boards’ decisions, we expected the 
Department to have made every effort to ensure that the boards had the 
resources to make informed decisions. Such resources are not only financial 
but also include appropriate guidance to ensure that the boards are well 
versed in the most appropriate techniques and approaches needed to carry 
out their tasks. They also include the kind of policy directives discussed 
earlier and clear direction on their roles and responsibilities. However, we saw 
no evidence that the Department carried out this responsibility. 


6.55 The lack of clarity in the roles and responsibilities is evident in a letter 
the Minister wrote the Review Board to question the relevance of some of its 
recommendations and to indicate the need to define roles and responsibilities 
as well as a process for consultation on future projects. He wrote that in all 
future environmental assessments, the Review Board should explain how 
each recommendation would prevent the environmental impacts identified in 


the assessment. 


6.56 When the boards were created, the federal government left them to 
determine how they would conduct their business. It provided no systematic 
orientation programs for board members and staff so they would understand, 
for example, the powers and procedures of federal boards with administrative 
tribunal responsibilities, the responsibility of federal boards to comply with 
federal requirements such as contracting policies, the various laws that 
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Renewing the Department’s role 


govern resource management in the NWT, and the extent of board members’ 
responsibilities. Currently, the Minister’s letter of offer to newly appointed 
board members says nothing about their responsibilities and duties as board 
members; it comments only on the requirement to comply with a code of 
conduct. 


6.57 We noted that the September 2004 report to the government by the 
External Advisory Committee on Smart Regulation, Smart Regulation: A 
Regulatory Strategy for Canada, addressed some of these issues. 


6.58 We believe that Indian and Northern Affairs can help the boards to 
carry out their functions if it develops a way for boards to share information 
with each other regularly and receive updates on federal expectations and 
recent legal rulings and interpretations. Yet the Department has not made 
sufficient effort to ensure that a means of sharing information is in place. 


6.59 Recommendation. Indian and Northern Affairs Canada should work 
with the boards under the Mackenzie Valley Resource Management Act to 
identify best practices and to assess training needs and provide for them, 
where appropriate. 


6.60 Recommendation. Indian and Northern Affairs Canada should work 
with the boards under the Mackenzie Valley Resource Management Act and 
other boards in the Northwest Territories to develop a permanent process for 
sharing best practices and solutions to the challenges they face. 


Department’s response. The Department has already met with some boards 
to discuss outstanding issues (for example, best practices, training needs, etc.) 
and has developed a process, which includes the NWT Board Forum, for 

ongoing dialogue to resolve those issues. This will become an ongoing agenda 


item at the next NWT Board Forum, scheduled for fall 2005. 


The Department will research and compile, as a starting point, best practices 
of other institutions of public government or expert organizations. The 
Department will prepare a preliminary report by fall 2005. 


The boards and government will utilize the NWT Board Forum as a key 
vehicle for discussing best practices and to assess training needs. The Board 
Forum meets regularly during each year. The Department expects that 
changes to the boards’ operations resulting from these discussions will start to 
be reflected in 2006-07 strategic, business, and expenditure plans of the 
boards. 


The Department needs to hold the boards accountable for managing the process 


6.61 The Mackenzie Valley Land and Water Board, the Sahtu and the 
Gwich’in land and water boards, and the Mackenzie Valley Environmental 
Impact Assessment Board were all created by federal legislation and are 
wholly funded by the federal government. The Minister for Indian and 
Northern Affairs also appoints all board members. 


6.62 The boards are required to produce audited financial statements of 
their operations each year, and they do. Beyond that, the annual reports of 
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each board contain little information to demonstrate the board’s 
accountability for managing their responsibilities in the best interests of the 
residents of the Mackenzie Valley and all Canadians. Nor has the Department 
requested that they do so. 


6.63 The Department has the responsibility to request such accountability 
reporting from the boards, because the federal government funds them 
through flexible transfer agreements. Under the Policy on Transfer Payments, 
the federal government calls for a results-based management and 
accountability framework. Specifically, it calls for a written agreement 
between the department and the recipient that identifies the expected 
results. Furthermore, it requires the recipient to account for and report on the 
results actually achieved. It also identifies the flexible transfers Indian and 
Northern Affairs uses for funding the boards as being covered by this policy. 


6.64 These boards are not accountable for decisions they make when acting 
in a quasi-judicial capacity. 


6.65 One vehicle for demonstrating accountability for results is an annual 

report. A good annual report provides information that stakeholders can use 
to hold management accountable for performance against the organization’s 
responsibilities. A good report also indicates what is working and what is not. 


6.66 The Smart Regulation report provides further support for such an 
accountability framework: 


When taking regulatory action, regulators should announce the 
results they wish to attain, the manner in which they intend to 
measure them, as well as when and at what frequency they will 
report on them. They must demonstrate their progress in 
achieving these results and be prepared to modify their 
approach if necessary. Evidence of performance is essential to 
sustain public trust. 


6.67 Annual reporting to the Minister could be, for example, the vehicle 
whereby a board establishes and reports on the kind of service any applicant 
or intervener should expect to receive and how well the board is meeting 
those standards. In effect, it could become an accountability report that 
includes information on finances and on the way the board manages its 
responsibilities for the process. 


6.68 Recommendation. Indian and Northern Affairs Canada should 
require that boards include in their annual reports to the Minister 
information not only on the board’s financial performance but also on the 
way they manage their responsibilities for the process. 


Department’s response. All boards currently report on their financial 


performance annually. 


The Department will continue discussions with the boards to implement 
changes to their reporting requirements to reflect not only their financial 
performance but also on the way in which they manage responsibilities for the 
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Chapter 6 


process. Changes to the boards’ reporting documentation will be evident by 


the 2005-06 reports. 


This initiative will be linked to the development or improvement of strategic 
plans. 


6.69 Recommendation. Indian and Northern Affairs Canada should 
require that reporting on financial and non-financial performance begin with 
the annual reports for 2005—06 and the Minister should make the reports 
public. 


Department’s response. Discussions regarding changes to the reporting 
requirements are already underway. The Department will work with the 
boards to expand and strengthen the content of the annual reports. Initial 
changes will be evident in time to be reflected in the 2005-06 annual reports. 


Good reporting begins with a clear understanding of the accountability relationship 


6.70 Before the boards can develop appropriate accountability reports, there 
is aneed for clear direction from the government on the roles and 
responsibilities of the boards. 


6.71 The Act provides a start; it indicates that the objective of the 
Mackenzie Valley Land and Water Board is to regulate land and water use to 
provide optimum benefit to residents of the Mackenzie Valley and to all 
Canadians. It also specifies that the Mackenzie Valley Environmental Impact 
Review Board is to protect the environment from significant adverse impacts 
of development and to protect the social, cultural, and economic well-being 
of people in the Valley. 


6.72 These objectives represent high-level outcomes of the Act. They are 
primarily the responsibility of the Department and are difficult to report 
against except through periodic evaluations. Moving from these high-level 
intentions to an operational level means developing a working management 
framework for the boards’ operations. Such a management framework would 
require a clear statement of the boards’ roles and responsibilities. It also 
would require strategic plans that include annual operational plans that 
describe how the boards are to carry out their responsibilities and appropriate 
indicators to hold them accountable for doing so. 


6.73. The American Society for Quality defines strategic planning as “the 
process by which an organization envisions its future and develops strategies, 
goals, objectives, and action plans to achieve that future.” We expected that 
with the creation of these boards, or relatively soon after, the Department 
would have provided direction on their roles and responsibilities and the role 
of strategic planning in reporting performance. But this was not the case. 


6.74 We found that the Review Board, on its own initiative, has made some 
progress by developing a strategic plan that includes possible performance 
measures and recognizes the need to develop service standards. 


6.75 Demonstrating performance could include establishing and reporting 
against service standards. It could also include working with such 
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organizations as the American Society for Quality or the International 
Organization for Standardization to develop service standards and 
demonstrate levels of quality achieved. In this regard, we do not mean 
measures that relate to any quasi-judicial functions of the boards. 


6.76 Recommendation. Indian and Northern Affairs Canada, in 
consultation with the boards under the Mackenzie Valley Resource Management 
Act, the Aboriginal communities in the Northwest Territories, and other 
stakeholders, should clarify the roles and responsibilities of the boards. 


Department’s response. Bilateral discussions on roles and responsibilities 
with some of the boards are already underway and replies to our invitation 
from the others are pending. In addition, this will become an agenda item for 
the NWT Board Forum. The Department will also initiate discussions with 
the representatives of groups with settled claims to ensure that roles and 
responsibilities reflect the claims agreements and legislation. This is an 
ongoing process of updating, renewal, and evolution. First results will be 


evident by April 2006. 


6.77 Recommendation. Indian and Northern Affairs Canada should work 
with each board under the Mackenzie Valley Resource Management Act to 
develop a strategic plan that includes a statement about the board’s mandate, 
vision, and mission; strategies for achieving them; and measures to 
demonstrate performance. 


Department’s response. Discussions with some boards on the development 
of, or strengthening existing, strategic plans is already under way and will 
continue. Other boards will be contacted for bilateral discussions. In addition, 
this will become an ongoing agenda item for the NWT Board Forum. All the 
boards will be requested to develop a strategic plan by April 2006. 


The Department recognizes that strategic plans and performance 
measurements are not static and improvements will be ongoing. 


6.78 Recommendation. Indian and Northern Affairs Canada should 
include in its Report on Plans and Priorities for 2005-06 a section that indicates 
how it plans to address the recommendations in this chapter. In subsequent 
performance reports, it should demonstrate its performance against these 


plans. 


Department’s response. The Department will include in its Report on Plans 
and Priorities (RPP) an action/work plan that indicates how it plans to 
address the recommendations in this chapter and report on progress. The 
action/work plan will be completed by April 2006. Future RPPs will report 
progress and achievements. 


The Department needs to establish an effective working relationship with the boards 


6.79 We observed little formal communication between the boards and the 
Department to identify common challenges and find common solutions, as 
reported earlier in the chapter. We saw little indication that the Department, 
whose Minister has the authority to consult with the boards in providing 
policy direction, has worked with the boards to develop a shared 
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interpretation of key terms in the legislation. We saw no indication that the 
Department has made any effort to establish reasonable standards for water 
quality. We also saw no evidence of any kind of accountability reporting 
except for financial reporting. 


6.80 Indian and Northern Affairs Canada has a responsibility to improve 
the state of resource management in the Northwest Territories. In our view, 
the Department needs to strengthen its relations with the boards in order to 
meet its responsibilities over the coming decade. This includes exercising its 
authority under the Mackenzie Valley Resource Management Act and consulting 
more closely with the boards. 


6.81 We noted that the Department has made some effort in the last two 
years to improve its relationship with the boards. For example, it established a 
Board Relations Secretariat in Yellowknife. The Secretariat’s purpose is to 
improve communication between the Department and the boards, to help 
resolve operational issues and interpretation differences that arise in 
implementing the Act, to administer board appointments by the Minister and 
funding arrangements for the regional boards, and to provide day-to-day 
advice and support to the boards. 


6.82 We saw, however, no evidence of the Department going beyond 
developing a relationship at the operational level. For example, we saw little 
effort by the Department to develop an ongoing relationship among the 
chairs and executive directors of the boards and senior officials in the 
Northern Affairs branch of the Department. 


6.83 Recommendation. Indian and Northern Affairs Canada should 
establish an ongoing process of consultation between the heads of the boards 
under the Mackenzie Valley Resource Management Act and the senior officials 
of the Department. 


Department’s response. The Department has requested that the boards 
increase and regularize their consultation with the government on key issues 
and will undertake bilateral meetings as required. In addition, the NWT 
Board Forum will be utilized as a key vehicle for ongoing consultation with 
the heads of the boards and senior departmental officials. 


Conclusion 


6.84 We believe that Indian and Northern Affairs Canada is not adequately 
managing its responsibilities that form a key part of the process for approving 
the development of non-renewable resources in the Northwest Territories. 


6.85 It has not yet exercised its authority under the Mackenzie Valley 
Resource Management Act to provide adequate direction to the public boards 
that manage the application process for the development of renewable and 
non-renewable resources in the Mackenzie Valley. Such direction would 
ensure that the details to make the process run smoothly are in place. These 
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include standards for water quality and guidance on key terms in the 
legislation. 


6.86 The Department has not taken steps to ensure that the boards have 
the appropriate management foundation and ongoing assistance to help them 
carry out their responsibilities. 


6.87 The Department also has not obliged the boards to comply with the 
requirement to be accountable for managing their responsibilities under the 
process in the best interests of the residents of the Mackenzie Valley and all 
Canadians. 


6.88 These matters are important for two reasons. First, while we recognize 
that the boards and the federal government need time to iron out their 
working relationships in this relatively new process, the continued absence of 
policy direction in crucial areas, the lack of support for effective knowledge 
building, and the lack of an appropriate accountability model have put the 
investment climate in the Northwest Territories at risk. Second, in the 
current negotiations to devolve federal responsibilities to the territories, 
control over non-renewable resources is high on the agenda. 


6.89 In the Indian and Northern Affairs Canada and Canadian Polar 
Commission Performance Report for the period ending March 31, 2003 the 
Department noted that the investment climate in the North is influenced by 
the efficiency, transparency, and fairness of regulatory frameworks and the 
new powers of public institutions to manage land and resources. It also noted 
the concerns expressed by industry that uncertainty, instability, and 
inefficiencies are constraining investment and limiting business opportunities. 
We believe that implementing our recommendations will contribute to 
remedying these concerns. 
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About the Audit 


Objectives 


The objective of the audit was to determine how well the federal government is managing its responsibilities 
associated with the process for the development of non-renewable resources in the Northwest Territories, other than 


the Inuvialuit Settlement Region. 


Scope and approach 


We audited the process that was established by the Mackenzie Valley Resource Management Act (1998), and focussed 
mainly on Indian and Northern Affairs Canada. 


In particular, we looked INAC’s responsibilities that form part of the process beginning with the application for a 
land permit and/or water licence and ending with the decision on the application. 


We conducted the audit mainly through interviews of officials and review of documents from the departments 
involved, primarily Indian and Northern Affairs Canada. We also reviewed publicly available documents associated 
with resource development applications. In addition, we interviewed key stakeholders, including representatives of 
the mining companies and industry, the Government of the Northwest Territories, and Aboriginal groups. 


This process for development involves four boards: the Mackenzie Valley Land and Water Board; its two panels, the 
Gwich’in Land and Water Board and the Sahtu Land and Water Board; and the Mackenzie Valley Environmental 
Impact Review Board. We did not audit any of the boards’ practices, procedures, or internal administration 
associated with their responsibilities; nor did we examine the roles that other federal departments and agencies play 
in the process. We did however interview officials from the boards to understand how Indian and Northern Affairs is 
managing its responsibilities. 


Criteria 


We expected that Indian and Northern Affairs Canada would manage the process by the following: 


* communicating the process to stakeholders in a way that is timely, transparent, understandable, and 
predictable; 


* conducting periodic reviews and making adjustments where necessary; 

* ensuring that the process has timelines that are clear, managed, and reviewed; 

* ensuring that the process has service standards that are clear, managed, and reviewed; and 

* developing and managing a risk management process. 
We expected that Indian and Northern Affairs Canada would manage the need for appropriate capacity in all steps, 
including 


* conducting periodic reviews to determine if there are any gaps in resources needed to carry out its 
responsibilities, and 


* preparing and implementing a plan for filling any gaps. 


We expected that Indian and Northern Affairs Canada would ensure accountability for the organizations involved in 
the process, including having 


+ aclear understanding by all federal entities involved in the process of their roles, responsibilities, and 
accountability relationships; 


* aclear leadership role for the process in the government; 
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* an appropriate accountability framework between the various organizations involved in the process and with 
Parliament; 


* aclear understanding by participants that the process is fair; and 


* a process for assessing performance. 


Audit team 


Assistant Auditor General: Ron Thompson 
Principal: Jeff Greenberg 
Director: Martin Ruben 


Stacey Wowchuk 
Lena Zecchino 


For information, please contact Communications at (613) 995-3708 or 1-888-761-5953 (toll free). 
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